Motion detection device and method, video signal processing device and method and video display device

ABSTRACT

In a motion detection device that detects motion from two frames of a video signal, a pattern matching detector determines pattern similarity between pixel blocks centered on a pixel of interest in the two frames to detect pattern motion. An edge detector detects edge presence and direction in a vicinity of the pixel of interest. A frame difference detector generates a smoothed frame difference signal for the pixel of interest. The smoothing is carried out within appropriate extents selected according to the detected pattern motion and edge direction. A motion information corrector generates motion information for the pixel of interest from the frame difference signal. Appropriate selection of the smoothing extent reduces motion detection mistakes. The motion information is useful in motion adaptive video signal processing.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a motion detection device and methodfor use in motion adaptive signal processing of a video signal. Theinvention also relates to video signal processing devices and processingmethods using the above motion detection device and method, and to avideo display device using the processed video signal.

2. Description of the Related Art

Video signals are processed to enhance their displayed picture quality.The enhancement can be increased by detecting local image motion in theinput video signal and carrying out motion adaptive signal processing.This is done by switching adaptively between processing suitable forstill pictures and processing suitable for moving pictures, depending onwhether the picture is locally still or moving. The two main types ofmotion adaptive signal processing carried out on television videosignals are motion adaptive line interpolation and a type ofthree-dimensional (3D) noise reduction (NR) in which the strength of thenoise reduction is controlled according to motion. Both of these typesof signal processing are effective in improving picture quality.

Motion adaptive line interpolation is carried out when an interlacedtelevision signal is enhanced by conversion to progressive scanning. Inthe interlaced-to-progressive scan conversion process (also called IPconversion), when the picture is detected to be still, inter-fieldinterpolation is carried out by interleaving the lines of two temporallyconsecutive fields, thereby removing line flicker. When the picture isdetected to be moving, intra-field interpolation is carried out bygenerating an interpolated signal from adjacent scanning lines in thesame field.

In 3D noise reduction, each frame of the video signal is compared withthe signal from one or more preceding frames and signal components thatshow no correlation across the plurality of frames are eliminated asnoise. In infinite impulse response (IIR) noise reduction, thedifference between the signal of the current frame and the noise-reducedsignal of the preceding frame is multiplied by a coefficient andsubtracted from the signal of the current frame. In finite impulseresponse (FIR) noise reduction, the noise is reduced by a filteringprocess in which the video signals of a plurality of temporallydiffering frames are multiplied by coefficients and added together. Bothmethods effectively remove noise components, which are uncorrelated onthe time axis, from parts of the picture without motion. When motion ispresent, however, trails and ghosts appear and edges become blurred. Toprevent these problems, a motion adaptive process is carried out bylimiting the degree of noise reduction according to the amount of motiondetected.

An effective way to obtain a motion signal indicating the amount ofmotion for motion adaptive signal processing purposes is to obtain aframe difference signal (frame-to-frame difference signal) by takingdifferences between the current input video signal (the current framesignal) and the signal of the immediately preceding frame (the precedingframe signal). When motion is detected only from frame-to-framedifferences, however, detection errors occur. Part of a still picturemay be incorrectly detected as moving, or part of a moving picture maybe incorrectly detected as still. When motion adaptive signal processingis carried out, the accuracy of motion detection has a major impact onthe quality of the displayed picture, so that motion detection processesthat avoid such motion detection errors have been proposed.

In one of these proposed methods, motion is detected from the framedifference signal, after passing the frame difference signal through avertical band-limiting low-pass filter (a vertical LPF), or using thespatial filter having a greater tap length vertically than horizontally,so as to carry out motion adaptive signal processing tailored to thespatial shape of the picture (see, for example, FIGS. 1 and 3 inJapanese Patent No. 2596166).

By applying a vertical filtering process to the frame difference signalor by detecting motion with a spatial filter having a longer verticaltap length as described above, the proposed method obtains a motiondetection signal.

There is still the problem, however, that the frame-to-frame differencesdo not just reflect motion; they also include a noise component.Particularly at edges in the picture, the motion component and the noisecomponent are intermixed, causing pixels that are actually still to besometimes misrecognized as moving.

Moreover, when there is a slowly moving object or a moving object with arepetitive pattern of vertical or horizontal lines, calculation ofdifferences between filtered frames can fail to detect motion, causingthe picture to be incorrectly classified as still, because thecalculated differences may be very small.

Thus in a picture that is even slightly unsteady or includes noise,frame differences due to the unsteadiness or noise affect motiondetection by causing still pictures to be misrecognized as moving, andin a picture with slowly moving objects or moving objects withrepetitive patterns, filtering of the frame difference signal can makeit impossible to obtain values from which motion can be detected, sothat the motion fails to be recognized.

These motion detection errors lead to degraded picture quality bycausing flicker, blur, combing, and other problems. Combing is a problemin which the picture splits up in a comb-like pattern.

The present invention addresses the above problems with the object ofproviding a motion detection device and method that can perform highlyaccurate motion detection, without mistakenly detecting still parts asmoving or moving parts as still, thereby reducing such forms of picturequality degradation as flicker, blur, and combing in the results ofmotion adaptive signal processing, and with the further objects ofproviding a video signal processing device, a video signal processingmethod, and a video display device using the above motion detectiondevice and method.

SUMMARY OF THE INVENTION

The invention provides a motion detection device for detecting motion ina video signal including temporally differing first and second frames.The motion detection device includes a pattern matching detector, anedge detector, a frame difference detector, and a motion informationcorrector.

The pattern matching detector calculates pattern similarity between apixel block in the first frame and a pixel block in the second frame andgenerates a moving block coefficient indicating block movement based onthe similarity. The pixel block in the first frame is centered on apixel of interest at which motion is to be detected. The pixel block inthe second frame is positioned at a pixel position corresponding to thepixel of interest.

The edge detector detects edges in a vicinity of the pixel of interestfrom the video signal in the first and second frames, and generates atleast one edge decision coefficient indicating a degree of edgepresence.

The frame difference detector uses the moving block coefficient and theedge decision coefficient to select an extent of horizontally alignedpixels including the pixel of interest or an extent of verticallyaligned pixels including the pixel of interest, performs a smoothingprocess within the selected extent, and performs a frame-to-framedifference calculation before or after the smoothing process, therebygenerating a frame difference signal for the pixel of interest.

The motion information corrector generates motion information for thepixel of interest from the frame difference signal generated by theframe difference detector.

The invention also provides a video signal processing device forperforming motion adaptive scanning line interpolation based on a motiondetection signal output by the above-mentioned motion detection deviceto convert an interlaced scanning video signal to a progressive videosignal. The video signal processing device includes a motion adaptiveinterpolator and a rate doubler. The motion adaptive interpolatorreceives the motion detection signal output from the motion detectiondevice and generates a scanning line interpolation signal responsive tothe result of motion detection for each pixel. The rate doubler uses thescanning line interpolation signal generated by the motion adaptiveinterpolator to generate the progressive video signal.

The invention provides another video signal processing device forperforming three dimensional noise reduction based on the motiondetection signal output by the above-mentioned motion detection deviceto eliminate noise components lacking frame-to-frame correlation fromthe video signal. This video signal processing device includes a motionadaptive noise reducer that uses the motion detection signal to controlthe noise reduction effect.

The invention also provides a video display device including either ofthe above-mentioned video signal processing devices, a display unit fordisplaying a video picture, and a display processor for causing thedisplay unit to display the video picture responsive to a video signaloutput by the video signal processing device.

The motion detection device according to the invention can detect motionwith high accuracy, so that still parts of the picture are notincorrectly detected as moving and moving parts are not incorrectlydetected as still.

By using this motion detection result, the video signal processingdevices according to the invention can perform motion adaptiveprocessing without picture degradation due to flicker, blur, or combing.

By displaying a video signal processed by either of the above-mentionedvideo signal processing devices, the video display device according tothe invention can display a video picture of high quality.

BRIEF DESCRIPTION OF THE DRAWINGS

In the attached drawings:

FIG. 1 is a block diagram showing an exemplary configuration of a videosignal processing device in a first embodiment of the invention;

FIG. 2A is a block diagram showing an exemplary configuration of thepattern matching detector used in the video signal processing device inthe first embodiment;

FIG. 2B is a block diagram showing an exemplary configuration of thematching vertical high frequency component generator used in the patternmatching detector in FIG. 2A;

FIG. 2C is a block diagram showing an exemplary configuration of thematching horizontal high frequency component generator used in thepattern matching detector in FIG. 2A;

FIG. 2D is a block diagram showing an exemplary configuration of thefirst moving block coefficient converter used in the pattern matchingdetector in FIG. 2A;

FIG. 2E is a block diagram showing an exemplary configuration of thesecond moving block coefficient converter used in the pattern matchingdetector in FIG. 2A;

FIG. 2F is a block diagram showing an exemplary configuration of thethird moving block coefficient converter used in the pattern matchingdetector in FIG. 2A;

FIG. 3 shows pixels constituting an exemplary pixel block used inpattern matching in the pattern matching detector in the firstembodiment;

FIGS. 4A and 4B show other examples of pixel blocks that may be used inpattern matching in the pattern matching detector in the firstembodiment;

FIG. 5 illustrates positional and temporal relationships andrelationships between pixels in the current frame signal and the delayedsignals when the video signal processing device in the first embodimentreceives an interlaced input signal;

FIGS. 6A and 6B illustrate positional relationships between pixels inthe current frame signal and the one-frame delayed signal in the firstembodiment;

FIGS. 7A and 7B show exemplary processing by which the matching verticalhigh frequency component extractor in FIG. 2B determines the matchingvertical high frequency component vebpm;

FIGS. 8A and 8B show another example of the processing by which thematching vertical high frequency component extractor in FIG. 2Bdetermines the matching vertical high frequency component vebpm;

FIG. 9 shows an exemplary conversion curve by which the pattern matchingdetector used in the video signal processing device in the firstembodiment converts block matching results to moving block coefficients;

FIG. 10 is a block diagram showing an exemplary configuration of theedge detector in the video signal processing device in the firstembodiment;

FIG. 11 shows an exemplary conversion curve by which the edge detectorused in the video signal processing device in the first embodimentconverts edge detection results to edge coefficients;

FIG. 12 is a block diagram showing an exemplary configuration of theframe difference calculator in the frame difference detector used in thefirst embodiment;

FIG. 13 is a block diagram showing an another exemplary configuration ofthe frame difference calculator in the frame difference detector used inthe first embodiment;

FIG. 14 shows an exemplary input-output characteristic of the nonlinearconversion unit in the frame difference detector used in the firstembodiment;

FIG. 15 is a block diagram showing an exemplary configuration of themotion information corrector used in the video signal processing devicein the first embodiment;

FIG. 16 shows an exemplary input-output characteristic of the motiondetection signal converter used in the video signal processing device inthe first embodiment;

FIGS. 17 and 18 constitute a flowchart illustrating the operation of themotion detection device used in the video signal processing device inthe first embodiment;

FIG. 19 is a block diagram showing an exemplary configuration of a videosignal processing device in a second embodiment of the invention;

FIG. 20A is a block diagram showing an exemplary configuration of thepattern matching detector used in the video signal processing device inthe second embodiment;

FIG. 20B is a block diagram showing an exemplary configuration of thematching vertical high frequency component generator used in the patternmatching detector in FIG. 20A;

FIG. 20C is a block diagram showing an exemplary configuration of thematching vertical high frequency component generator used in the patternmatching detector in FIG. 20A;

FIG. 20D is a block diagram showing an exemplary configuration of thematching horizontal high frequency component generator used in thepattern matching detector in FIG. 20A;

FIG. 20E is a block diagram showing an exemplary configuration of thematching horizontal high frequency component generator used in thepattern matching detector in FIG. 20A;

FIG. 21 is a block diagram showing an exemplary configuration of amotion adaptive processor used as the video signal processing device ina third embodiment of the invention;

FIG. 22 is a block diagram showing another exemplary configuration of amotion adaptive processor used as the video signal processing device inthe third embodiment;

FIG. 23 is a block diagram showing an exemplary configuration of amotion adaptive processor used as the video signal processing device ina fourth embodiment of the invention;

FIGS. 24 and 25 are block diagrams showing other exemplaryconfigurations of a motion adaptive processor used as the video signalprocessing device in the fourth embodiment; and

FIG. 26 is a block diagram showing an exemplary configuration of a videodisplay device in a fifth embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

The motion detection device according to the invention detects motion ina video signal from temporally differing first and second frames of thevideo signal. The video signal processing devices according to theinvention use the above-mentioned motion detection device to process avideo signal. The display device according to the invention displays avideo signal processed by the above-mentioned video signal processingdevice.

The various units described in this application may be implemented aselectrical circuits (hardware), or as software, in the form of aprogrammed computing device.

First Embodiment

FIG. 1 is a block diagram showing the configuration of a video signalprocessing device of a first embodiment of the invention (device forimplementing the video signal processing method of the first embodimentof the invention). The device is configured to detect image motion pixelby pixel from the video signals of a first frame and a temporallydiffering second frame: for example, from the video signal of thecurrent frame and the video signal the preceding frame.

The video signal processing device of the first embodiment in FIG. 1sequentially receives an interlaced input video signal Di0 indicatingthe values of the individual pixels constituting an input video picture.As shown in FIG. 1, the video signal processing device includes a framedelay unit 13 and a motion detection device 1.

The frame delay unit 13 includes first and second field memories 11, 12that output a signal d1 f delayed by one field (a one-field delayedsignal) and a signal d2 f delayed by one frame (a one-frame delayedsignal), respectively. The first and second field memories 11, 12function as field delay units, outputting the one-field delayed signald1 f and one-frame delayed signal d2 f by delaying the input videosignal by one field each. Fields are delimited by successive verticalsynchronizing signals in the interlaced signal.

The input video signal Di0, the one-field delayed signal d1 f, and theone-frame delayed signal d2 f span two temporally differing frames. Inthe detailed examples that follow, the input video signal Di0 may bereferred to as the current frame signal or the input frame signal, theone-frame delayed signal d2 f as the signal of the preceding frame, andthe one-field delayed signal d1 f as the signal of the preceding field.

The motion detection device 1 uses the input video signal (current framesignal) Di0, the one-frame delayed signal (the signal of the precedingframe) d2 f, and the one-field delayed signal (the signal of thepreceding field) d1 f to output a motion detection signal mds thatindicates degrees of motion based on frame-to-frame differences.

The motion detection device 1 includes a spatial and temporal expansionfiltering unit 16, a motion detection signal converter 19, a framedifference detector 20, pattern matching detector 30, an edge detector40, and a motion information corrector 50.

The pattern matching detector 30 performs a pattern matching operationin which it calculates a pattern similarity between two pixel blocks,one in the current frame signal Di0, the other in the one-frame delayedsignal d2 f. This operation generates first and second moving blockcoefficients dfmat, cbmat indicative of pixel block motion.

The edge detector 40 receives the current frame signal Di0, theone-frame delayed signal d2 f, and the one-field delayed signal d1 f,detects edges in the image in the vicinity of a pixel of interest in thevideo signal, and outputs first and second edge decision coefficientsefkm, cedkm.

The frame difference detector 20 receives the current frame signal Di0and the one-frame delayed signal d2 f, selects a smoothing extentresponsive to the first moving block coefficient dfmat from the patternmatching detector 30 and the first edge decision coefficients efkm fromthe edge detector 40, performs a smoothing process within the selectedextent, detects a smoothed frame-to-frame difference, and outputs aframe difference signal fdiff representing the difference between theframes at the same pixel.

The motion information corrector 50 modifies the frame difference signalfdiff from the frame difference detector 20, responsive to the secondmoving block coefficients cbmat from the pattern matching detector 30and the second edge decision coefficients cedkm from the edge detector40, to obtain motion information md0.

The spatial and temporal expansion filtering unit 16 performs processessuch as spatial and temporal filtering and isolated point removal on themotion information signal md0, modifies the motion information in thespatial and temporal directions, and outputs a motion signal afmd. Themotion detection signal converter 19 converts the motion signal afmdfrom the spatial and temporal expansion filtering unit 16 to the motiondetection signal mds indicating the degree of motion in the videosignal.

The frame difference detector 20 includes a frame difference calculator21 and a nonlinear conversion unit 22. The frame difference calculator21 switches the smoothing extent responsive to the moving blockcoefficient dfmat from the pattern matching detector 30 and edgedecision coefficients efkm from the edge detector 40 and calculates aframe difference frd between the current frame signal and the one-framedelayed signal. The nonlinear conversion unit 22 performs a nonlinearconversion on the value of the frame difference to obtain a differencesignal representing motion and outputs it as the frame difference signalfdiff.

The spatial and temporal expansion filtering unit 16 includes a temporalfiltering section 17, which modifies the motion information signal md0by performing a filtering process that stretches the motion informationin the temporal direction and outputs the modified signal fmd0, and aspatial filtering section 18, which modifies this signal by performingprocesses such as a filtering process that stretches the motioninformation in the horizontal or vertical spatial direction and aprocess to remove isolated points. The modified signal from the spatialfiltering section 18 is output as the motion signal afmd.

In the description below, the first embodiment detects motion in thevideo signal between the current frame signal and the one-frame delayedsignal.

As noted above, the first and second field memories 11, 12 constitutingthe frame delay unit 13 are memory devices that output the video signalwith a one-field delay. The first field memory 11 delays the input videosignal Di0 by one field and outputs the one-field delayed signal d1 f.The second field memory 12 delays the one-field delayed signal d1 f fromthe first field memory 11 by one field and outputs a two-field delayedsignal (the one-frame delayed signal) d2 f.

Since the input video signal Di0 is interlaced, although the pixels ofthe one-frame delayed signal d2 f output from the frame delay unit 13are disposed in the same positions as the pixels in the input videosignal Di0, the pixels in the one-field delayed signal d1 f output fromthe first field memory 11 are half a line above or below the pixels inthe input video signal Di0. The input video signal Di0 and the one-framedelayed signal d2 f are thus mutually in phase while the input videosignal Di0 and the one-field delayed signal d1 f are mutually out ofphase.

The input video signal Di0 is input to the motion detection device 1 asthe current frame signal, while the one-frame delayed signal d2 f andone-field delayed signal d1 f from the frame delay unit 13 are input asthe signals of the preceding frame and preceding field. The motiondetection device 1 detects motion of the video signal for each pixelbetween the current frame signal Di0 and the signal of the precedingframe, and outputs the motion detection signal mds indicating the degreeof motion from the detection results.

The motion detection device 1 switches the extent of the smoothingprocessing performed in the determination of the difference betweenframes responsive to pattern motion and edge direction in the image,using the moving block coefficients obtained from pattern matching ofpixel blocks in the current frame signal Di0 and the one-frame delayedsignal d2 f and the edge decision coefficients obtained by edgedetection, thereby obtaining a frame difference signal that has beensmoothed within an appropriate extent. The motion detection device 1then further modifies the frame difference signal responsive to patternmotion and edge directions, and generates motion information modified soas to facilitate motion detection at edges and in patterns where movingareas tend to be misrecognized as still. These modifications preventmotion from being mistakenly detected or missed and enable the motiondetection device 1 to output a motion detection signal mds indicatingthe degree of motion with high accuracy.

The current frame signal Di0 and one-frame delayed signal d2 f are inputto the pattern matching detector 30 in the motion detection device 1.

The pattern matching detector 30 performs a pattern matching operationthat calculates the pattern similarity between two pixel blocks, one inthe current frame signal Di0, the other in the one-frame delayed signald2 f input as the signal of the preceding frame, and outputs the movingblock coefficients dfmat, cbmat that indicate pixel block motion. Theoutput moving block coefficients dfmat, cbmat are obtained from theresults of pattern matching performed for a pixel block centered on thepixel of interest at which motion is being detected; a large similarityvalue indicates different patterns in the pixel blocks, indicating thatthe image in the pixel blocks is moving, and could be described asindicating motion of the block pattern (referred to as a moving patternbelow) in a pixel block.

The pattern matching detector 30 is configured as shown, for example, inFIGS. 2A to 2E. The illustrated pattern matching detector 30 performspattern matching between pixel blocks in the one-frame delayed signal d2f and the current frame signal Di0, and generates moving blockcoefficients dfmat, cbmat indicating motion of the pixel block centeredon the pixel of interest P0 in the current frame signal Di0.

As shown, the pattern matching detector 30 includes a block matchingoperation unit 301, a matching vertical high frequency componentgenerator 308, a matching horizontal high frequency component generator309, and first, second, and third moving block coefficient conversionunits 311, 312, 313. As shown in FIG. 2B, the matching vertical highfrequency component generator 308 includes a horizontal block matchingoperation unit 303, and a matching vertical high frequency componentextractor 304. As shown in FIG. 2C, the matching horizontal highfrequency component generator 309 includes a vertical block matchingoperation unit 306, a matching horizontal high frequency componentextractor 307.

The range over which pattern matching is carried out by the patternmatching detector 30 is, for example, the block of nine pixels shown inFIG. 3, extending three lines vertically and three pixels horizontally(three rows and three columns), centered on the pixel of interest P0 inthe current frame Di0. This pixel block will be denoted b0 and will alsobe referred to as the pixel block of interest. Pattern matching iscarried out on this block b0 and a pixel block b2 (the reference pixelblock) centered on a pixel Pf2 (the central reference pixel) in the sameframe position in the one-frame delayed signal d2 f.

The pattern matching range is not limited to a block of nine pixels asshown in FIG. 3. In general, the block may include V lines verticallyand H pixels horizontally (V≧2, H≧2). For example it may be afifteen-pixel block including three lines vertically and five pixelshorizontally (three rows, five columns) as shown in FIG. 4A, or aneight-pixel block including two lines vertically and four pixelshorizontally (two rows, four columns) as shown in FIG. 4B. The morepixels are included in the pixel block, the broader the range over whichsimilarity can be calculated becomes, so that more stable values areobtained.

The more pixels there are and the broader the range on which thecalculation is performed becomes, however, the broader becomes the rangeaffected by pixel block motion.

FIG. 5 illustrates the temporal relation between the signals in the sameline in the current frame signal Di0 and the one-frame delayed signal d2f and positional relations in the vertical scanning direction (thedirection of higher and lower lines on the display screen) in eachframe. Also shown are the positional relationships of the pixel ofinterest P0 and pixel block of interest b0 on which pattern matching iscarried out in the pattern matching detector 30 and the correspondingcentral reference pixel Pf2 and reference block b2 in the frame in theone-frame delayed signal d2 f.

In FIG. 5, earlier frames are shown closer to the left edge, and pixelsin each frame displayed in higher positions on the screen are showncloser to the top. Whereas the current frame signal Di0 is the signal ofthe current frame (the latest frame), the one-field delayed signal d1 fis the signal of the frame one field before, the one-frame delayedsignal d2 f is the signal of the frame one frame before, and thetwo-frame delayed signal d4 f is the signal of the frame two framesbefore. The frame one frame before the current frame may be referred toas the immediately preceding frame. The frames are denoted by the samereference characters as the corresponding signals (Di0, d2 f, d1 f).

FIGS. 6A and 6B illustrate vertical and horizontal positionalrelationships between pixels in the current frame signal Di0 and theone-frame delayed signal d2 f. These drawings show relationships betweenthe pixel of interest P0 and the pixel block of interest b0 in thecurrent frame Di0 and the corresponding central reference pixel Pf2 andthe corresponding reference block b2 in the one-frame delayed signal d2f (immediately preceding frame d2 f).

The pattern matching detector 30 in FIG. 2A detects moving blockcoefficients indicating motion in the pixel block b0 centered on thepixel of interest P0 by pattern matching of pixel blocks between thecurrent frame signal Di0 and the signal d2 f of the frame one framebefore. In FIGS. 5, 6A, and 6B, the result of pattern matching betweenthe pixel block of interest b0 in the current frame Di0 and thereference pixel block b2 in the corresponding position in theimmediately preceding frame d2 f is obtained as a pattern similarity.

In FIGS. 2A to 2C, the current frame signal Di0 and the one-framedelayed signal d2 f input to the pattern matching detector 30 are inputto the block matching operation unit 301, horizontal block matchingoperation unit 303, and vertical block matching operation unit 306.

The block matching operation unit 301 extracts the pixel block ofinterest b0 (see FIGS. 5 and 6B) centered on the pixel of interest P0 inthe current frame Di0 and the reference pixel block b2 (see FIGS. 5 and6A) centered on the pixel Pf2 in the position corresponding to the pixelP0 in the immediately preceding frame d2 f, performs pattern matchingbetween the pixel blocks, and obtains a block matching quantity blkpmindicating the pattern similarity.

The pattern similarity between pixel blocks determined by patternmatching is obtained by calculating all absolute differences in pixelvalue between pixels in corresponding positions in the pixel blocks andobtaining a value based on the simple sum or a weighted sum of theabsolute values (also referred to as a sum of absolute differences) orby calculating a simple mean or weighted mean of the absolutedifferences.

More specifically, the block matching operation unit 301 calculates theabsolute difference in pixel value between the pixels in the pixel blockof interest b0 in the current frame Di0 and the pixels in thecorresponding positions in the reference pixel block b2 in theimmediately preceding frame d2 f, as given by equations (1) to (9)below:

d1=|P0a(−1)−Pf2a(−1)|  (1)

d2=|P0a(0)−Pf2a(0)|  (2)

d3=|P0a(1)−Pf2a(1)|  (3)

d4=|P0(−1)−Pf2(−1)|  (4)

d5=|P0−Pf2|  (5)

d6=|P0(1)−Pf2(1)|  (6)

d7=|P0b(−1)−Pf2b(−1)|  (7)

d8=|P0b(0)−Pf2b(0)|  (8)

d9=|P0b(1)−Pf2b(1)|  (9)

Next, a sum of these absolute values of differences d1 to d9 is obtainedand averaged as in equation (10) below. The resulting mean absolutedifference of the block is output as a block matching quantity blkpm.

blkpm=(d1+d2+d3+d4+d5+d6+d7+d8+d9)/9  (10)

In equation (10) above, to obtain the mean value of the nine pixels, thesum is divided by nine. The sum may be divided by a power of two such aseight (2³) or four (2²), however, for reasons of hardware configurationor ease of computation. The essential point is to obtain some sort ofindicator reflecting the mean value.

The block matching quantity blkpm calculated by equation (10) approaches‘0’ as the similarity between the pixel block of interest b0 and thereference pixel block b2 increases, indicating that the image is stillor nearly still with only a very small amount of motion. As thesimilarity decreases, the block matching quantity blkpm increases,indicating that the pixel blocks have different patterns, which meansthat the patterns are moving.

The block matching quantity blkpm calculated by the block matchingoperation unit 301 is output to the first, second, and third movingblock coefficient conversion units 311, 312, 313 in FIG. 2A as an indexof pattern similarity between the pixel block of interest b0 and thereference pixel block b2, or of pattern motion between the two pixelblocks.

Instead of a simple mean, a weighted mean of the absolute differences d1to d9 may be obtained. For example, the weighted mean may be obtained bytaking a weighted sum, where the weight of the differences increases asthe pixel approaches the central pixel P0 of the pixel block, anddecreases with increasing distance from the central pixel. For example,a block matching quantity blkpm based on a sum of absolute differencesmay be calculated as in equation (11) below, instead of equation (10).

blkpm=d1/8+d2/8+d3/8+d4/8+d5/2+d6/8+d7/8+d8/8+d9/8

The pixel block used in pattern matching need not be a nine-pixel blockas shown in FIG. 3; it may be a fifteen-pixel block as shown in FIG. 4Aor an eight-pixel block as shown in FIG. 4B. If the fifteen-pixel blockincludes three lines vertically and five pixels horizontally as shown inFIG. 4A, the block matching operation unit 301 extracts a pixel block ofinterest b0 and a reference pixel block b2, takes a sum of the absolutedifferences for the fifteen pixels, and calculates the mean value as theblock matching quantity blkpm.

Next, the horizontal block matching operation unit 303 carries outpattern matching on horizontally elongated sub-block areas constitutingparts of the pixel blocks in the respective frames, based on the currentframe signal Di0 and the one-frame delayed signal d2 f. For example,pattern matching is carried out on sub-blocks of three horizontallyaligned pixels in FIG. 3, and horizontal block matching quantities hp11,hp12, hp13 for three lines (three rows) are sent to the matchingvertical high frequency component extractor 304.

Like the block matching operation unit 301, the horizontal blockmatching operation unit 303 first obtains the absolute differences d1 tod9 in pixel value between the pixels in the pixel block of interest b0in the current frame Di0 and the pixels in the reference pixel block b2in the immediately preceding frame d2 f, as given by equations (1) to(9), and then obtains the sum of the absolute differences for each ofthe sub-blocks formed by dividing the pixel block b0 in the verticaldirection: for example, in three sub-blocks each having a size of oneline (one row) in the vertical direction. Since the pixel block sizemeasures three lines vertically and three pixels horizontally (threerows and three columns), sums of absolute differences are obtained herefor one-line-by-three-pixel (one-row-by-three-column) blocks, that is,for sub-blocks including three horizontally aligned pixels. Thehorizontal block matching quantities hp11, hp12, hp13 are given byequations (12) to (14) below.

hp11=d1+d2+d3  (12)

hp12=d4+d5+d6  (13)

hp13=d7+d8+d9  (14)

Since the absolute differences d1 to d9 in pixel values in the pixelblock are obtained in the block matching operation unit 301, theabsolute differences may be supplied from the block matching operationunit 301 to the horizontal block matching operation unit 303 and used tocalculate the horizontal block matching quantities hp11, hp12, hpl3.

One third of the sum of absolute differences on the right side inequations (12) to (14) may be taken to obtain the mean of the absolutedifferences, or the sum may be divided by four (a power of two) insteadof taking one third, for reasons of hardware configuration or ease ofcalculation.

The sum of the absolute differences obtained for each sub-block of threehorizontally aligned pixels may be a weighted sum. For example, theweight of the difference may increase as the pixel approaches thecentral pixel P0 or Pf2 of the pixel block, and decrease with increasingdistance from the central pixel. As a specific example, the horizontalblock matching quantity hp11 may be obtained from equation (15) belowinstead of from equation (12).

hp11=d1/4+d2/2+d3/4  (15)

From the input horizontal block matching quantities hp11, hp12, hp13,the matching vertical high frequency component extractor 304 obtains theabsolute differences in horizontal block matching quantity betweenadjacent lines (rows) and determines a matching vertical high frequencycomponent vebpm indicating the vertical high frequency component of theabsolute difference.

The horizontal block matching quantities hp11, hp12, hp13 are valuesbased on sums of the absolute differences. By obtaining the verticalhigh frequency component of the horizontal block matching quantities,that is, the vertical edge component, the similarity of patterns movingvertically can be obtained.

From the horizontal block matching quantities hp11, hp12, hp13 inputfrom the horizontal block matching operation unit 303, the matchingvertical high frequency component extractor 304 extracts a vertical highfrequency component by taking one-fourth of the absolute values ofline-to-line (row-to-row) differences of the horizontal block matchingquantities (absolute values of differences between the horizontal blockmatching quantities of the respective lines (rows) to obtain the valuesdh1 and dh2 given by equations (16) and (17) below, and outputs thegreater of the two values dh1, dh2 as a matching vertical high frequencycomponent vebpm, given by equation (18).

dh1=|hp11−hp12|/4  (16)

dh2=|hp12−hp13|/4  (17)

vebpm=MAX(dh1, dh2)  (18)

Instead of taking one-fourth of the absolute differences as in equations(16) and (17), one-half of the absolute differences may be taken.Alternatively, the maximum value of the absolute differences between thevalues of respective lines (rows) may be taken and divided by four. Ifthe horizontal block matching quantities hp11, hp12, hp13 are obtainedby averaging the absolute differences in the horizontal block matchingoperation unit 303, a similar value can be obtained without the divisionby four as indicated in the equations above. In any case, it issufficient if the value obtained reflecting the maximum of the absolutedifferences is obtained.

The matching vertical high frequency component vebpm calculated as inequation (18) is ‘0’ when the image is still, when the pixels in theblock have identical values and all the pixels in the block are movingin the same direction at the same speed, or when the pixels in the blockare moving in the horizontal direction at the same speed. Since theabsolute differences (horizontal block matching quantities hp11, hp12,hp13) increase as the similarity decreases, if the absolute differencehas a vertical high frequency component, the value of the matchingvertical high frequency component vebpm becomes large, indicating thatthe pixel block includes a component of motion in the verticaldirection.

For example, if the immediately preceding frame d2 f has an edge betweenthe first row and the second row as shown in FIG. 7A and if the currentframe Di0 has an edge between the second row and the third row as shownin FIG. 7B (that is, if the horizontal edge moves downward), thehorizontal block matching quantity hp11 is small, the horizontal blockmatching quantity hp12 is large, and the horizontal block matchingquantity hp13 is small, consequently the two values dh1, dh2 becomelarge, and the matching vertical high frequency component vebpm becomeslarge.

As another example, if the immediately preceding frame d2 f has an edgeat the top of the first row as shown in FIG. 8A and if the current frameDi0 has an edge between the first row and the second row (that is, ifthe horizontal edge moves downward), the horizontal block matchingquantity hp11 is large and the horizontal block matching quantitieshp12, hp13 are small; consequently, the dh1 value becomes large and thedh2 value becomes small, and the matching vertical high frequencycomponent vebpm becomes large.

The horizontal block matching operation unit 303 and the matchingvertical high frequency component extractor 304 shown in FIG. 2B incombination form the matching vertical high frequency componentgenerator 308 that generates the matching vertical high frequencycomponent vebpm by calculating a sum (simple or weighted) of theabsolute differences in pixel value between the pixels in correspondingpositions in the pixel blocks b0, b2, i.e., in corresponding pixelpositions in the current frame Di0 and the immediately preceding framed2 f, for each of a plurality of vertically divided sub-blocks of thepixel blocks b0 and b2, and taking differences between these sums invertically adjacent sub-blocks.

Although the pixel blocks on which pattern matching is carried out areconfigured as nine-pixel blocks in FIG. 3, the blocks may be configuredas shown in FIG. 4A or 4B. For example, if fifteen-pixel blocksincluding three lines vertically and five pixels horizontally as shownin FIG. 4A are used, the horizontal block matching operation unit 303carries out pattern matching on sub-blocks formed by five horizontallyaligned pixels and obtains horizontal block matching quantities forthree lines (three sub-blocks in different vertical positions).

More generally, if the pixel blocks b0, b2 include V lines verticallyand H pixels horizontally, the pixel blocks b0, b2 are dividedvertically into V sub-blocks, each formed by a single line of H pixels,in different vertical positions; the sum of the absolute differencesbetween sub-blocks in corresponding pixel positions in the current frameand the immediately preceding frame is obtained; then the matchingvertical high frequency component vebpm is obtained by takingdifferences between these sums in vertically adjacent sub-blocks.

The matching vertical high frequency component vebpm extracted by thematching vertical high frequency component extractor 304 is output tothe first and second moving block coefficient conversion units 311, 312as a value indicating the presence of a vertical motion component in thepixel blocks.

The vertical block matching operation unit 306 carries out patternmatching on vertically elongated sub-block areas constituting parts ofthe pixel blocks in the respective frames, based on the current framesignal Di0 and the one-frame delayed signal d2 f. For example, patternmatching is carried out on sub-blocks of three vertically aligned pixelsin FIG. 3, and vertical block matching quantities vp11, vp12, vp13 forthree pixels (three columns) are sent to the matching horizontal highfrequency component extractor 307.

The processing in the vertical block matching operation unit 306 isidentical to the processing in the horizontal block matching operationunit 303 except that the direction is vertical instead of horizontal.

Like the block matching operation unit 301, the vertical block matchingoperation unit 306 first obtains the absolute differences d1 to d9 inpixel value between the pixels in the pixel block of interest b0 in thecurrent frame Di0 and the pixels in the reference pixel block b2 in theimmediately preceding frame d2 f, as given by equations (1) to (9), andthen obtains the sum of the absolute differences for each of thesub-blocks formed by dividing the pixel block b0 in the horizontaldirection: for example, in three sub-blocks each having a size of onepixel (one column) in the horizontal direction. Since the pixel blocksize measures three lines vertically and three pixels horizontally(three rows and three columns), sums of absolute differences areobtained here for three-line-by-one-pixel (three-row by-one-column)blocks, that is, for sub-blocks including three vertically alignedpixels. The vertical block matching quantities vp11, vp12, vp13 aregiven by equations (19) to (21) below.

vp11=d1+d4+d7  (19)

vp12=d2+d5+d8  (20)

vp13=d3+d6+d9  (21)

As with the horizontal block matching operation unit 303, the absolutedifferences d1 to d9 obtained in the block matching operation unit 301may be supplied to the vertical block matching operation unit 306 andused to calculate the vertical block matching quantities vp11, vp12,vp13.

One third of the sum of absolute differences on the right side inequations (19) to (21) may be taken to obtain the mean of the absolutedifferences, or the sum may be divided by four (a power of two) insteadof taking one third, for reasons of hardware configuration or ease ofcalculation.

The sum of the absolute differences obtained for each sub-block of threevertically aligned pixels may be a weighted sum. For example, the weightof the difference may increase as the pixel approaches the central pixelP0 or Pf2 of the pixel block, and decrease with increasing distance fromthe central pixel. As a specific example, the vertical block matchingquantity vp11 may be obtained from equation (22) below instead of fromequation (19).

vp11=d1/4+d4/2+d7/4  (22)

From the input vertical block matching quantities vp11, vp12, vp13, thematching horizontal high frequency component extractor 307 obtains theabsolute differences in vertical block matching quantity betweenadjacent columns and determines a matching horizontal high frequencycomponent hebpm indicating the horizontal high frequency component ofthe absolute differences.

The vertical block matching quantities vp11, vp12, vp13 are values basedon sums of the absolute differences. By obtaining the horizontal highfrequency component of the vertical block matching quantities, that is,the horizontal edge component, the similarity of patterns movinghorizontally can be obtained (a similarity that increases in value ifthere is a horizontally moving pattern).

From the vertical block matching quantities vp11, vp12, vp13 input fromthe vertical block matching operation unit 306, the matching horizontalhigh frequency component extractor 307 extracts a horizontal highfrequency component by taking one-fourth of the absolute values ofpixel-to-pixel (column-to-column) differences of the vertical blockmatching quantities (absolute values of differences between the verticalblock matching quantities of the respective pixels (columns) to obtainthe values dv1 and dv2 given by equations (23) and (24) below, andoutputs the greater of the two values dv1, dv2 as the matchinghorizontal high frequency component hebpm, given by equation (25).

dv1=|vp11−vp12|/4  (23)

dv2=|vp12−vp13|/4  (24)

hebpm=MAX(dv1, dv2)  (25)

Instead of taking one-fourth of the absolute differences as in equations(23) and (24), one-half of the absolute differences may be taken.Alternatively, the maximum value of the absolute differences between thevalues respective pixels (columns) may be taken and divided by four. Ifthe vertical block matching quantities vp11, vp12, vp13 are obtained byaveraging the absolute differences in the vertical block matchingoperation unit 306, a similar value can be obtained without the divisionby four indicated as in the equations given above. In any case, it issufficient if the value obtained reflecting the maximum of the absolutedifferences is obtained.

The matching horizontal high frequency component hebpm calculated as inequation (25) is ‘0’ when the image is still, when the pixels in theblock have identical values and all the pixels in the block are movingin the same direction at the same speed, or when the pixels in the blockare moving in the vertical direction at the same speed. Since theabsolute differences (vertical block matching quantities vp11, vp12,vp13) increase as the similarity decreases, if the absolute differencehas a horizontal high frequency component, the value of the matchinghorizontal high frequency component hebpm becomes large, indicating thatthe pixel block includes a component of motion in the horizontaldirection.

The vertical block matching operation unit 306 and the matchinghorizontal high frequency component extractor 307 shown in FIG. 2C incombination form the matching horizontal high frequency componentgenerator 309 that generates the matching horizontal high frequencycomponent hebpm by calculating a sum (simple or weighted) of theabsolute differences in pixel value between the pixels in correspondingpositions in the pixel blocks b0, b2, i.e., in corresponding pixelpositions in the current frame Di0 and the immediately preceding framed2 f, for each of a plurality of horizontally divided sub-blocks of thepixel blocks b0 and b2, and taking differences between these sums inhorizontally adjacent sub-blocks.

Although the pixel blocks on which pattern matching is carried out areconfigured as nine-pixel blocks in FIG. 3, the blocks may be configuredas shown in FIG. 4A or 4B. For example, if fifteen-pixel blocksincluding three lines vertically and five pixels horizontally as shownin FIG. 4A are used, the vertical block matching operation unit 306carries out pattern matching on sub-blocks including three verticallyaligned pixels and obtains vertical block matching quantities over arange of five pixels horizontally (five sub-blocks in differenthorizontal positions).

More generally, if the pixel blocks b0, b2 include V lines verticallyand H pixels horizontally, the pixel blocks b0, b2 are dividedhorizontally into H sub-blocks, each including a single pixel from eachof V lines, in different horizontal positions; the sum of the absolutedifferences between sub-blocks in corresponding pixel positions in thecurrent frame and the immediately preceding frame is obtained; then thematching horizontal high frequency component hebpm is obtained by takingdifferences between these sums in horizontally adjacent sub-blocks.

The matching horizontal high frequency component hebpm extracted by thematching horizontal high frequency component extractor 307 is output tothe third moving block coefficient converter 313, as a value indicatingthe presence of a horizontal motion component in the pixel blocks.

The first, second, and third moving block coefficient conversion units311, 312, 313 have similar configurations. The first and second movingblock coefficient conversion units 311, 312 receive the block matchingquantity blkpm from the block matching operation unit 301 and thematching vertical high frequency component vebpm from the matchingvertical high frequency component extractor 304. The third moving blockcoefficient converter 313 receives the block matching quantity blkpmfrom the block matching operation unit 301 and the matching horizontalhigh frequency component hebpm from the matching horizontal highfrequency component extractor 307.

As shown in FIG. 2D, the first moving block coefficient conversion unit311 includes a block coefficient converters 302 a, a high frequencycomponent converter 305 a, and a coefficient combiner 314. As shown inFIG. 2E, the second moving block coefficient conversion unit 312includes a block coefficient converter 302 b, a high frequency componentconverter 305 b, and a coefficient combiner 315. As shown in FIG. 2F,the third moving block coefficient converter 313 includes a blockcoefficient converter 302 c, a high frequency component converter 305 c,and a coefficient combiner 316.

The first, second, and third moving block coefficient conversion units311, 312, 313 convert the block matching quantity blkpm received fromthe block matching operation unit 301 in accordance with the matchinghigh frequency component (vebpm or hebpm) to generate block coefficientsdfmat, dmat_mk, Vrdmat indicative of pixel block motion.

For example, the block coefficient converter 302 a in the first movingblock coefficient conversion unit 311 subtracts a given offset value M1from the block matching quantity blkpm, multiplies the difference by agiven magnification factor Mk1, performs a nonlinear conversion bylimiting the result ((blkpm−M1)×Mk1) to a given range (e.g., ‘0’ toMax_k), and outputs the resulting value as a block coefficient dmb11.

Alternatively, the nonlinear conversion may be carried out bymultiplying by the magnification factor Mk1 and then subtracting theoffset value M1 (blkpm×Mk1−M1), to obtain the block coefficient.

FIG. 9 shows an exemplary conversion curve by which the blockcoefficient converter 302 a converts the block matching quantity blkpm(on the horizontal axis) to the block coefficient dmb11 (on the verticalaxis) when the block coefficient dmb11 can take values from ‘0’ to Max_k(=1).

The block coefficient dmb11 is used as an indicator of the degree ofmotion of the block pattern in the pixel block.

The values of the block coefficient dmb11 from ‘0’ to ‘1’ may beconverted to values from ‘0’ to ‘8’ representing fractions ⅛, 2/8, andso on, with ‘8’ representing ‘1’.

In the example shown, when the block matching quantity blkpm is equal toor greater than a given value M2, the block coefficient dbmb11 takes themaximum value Max_k, indicating that the block is moving (the block istreated as definitely moving). When the block matching quantity blkpm isequal to or less than the offset value M1, the block coefficient dmb11is ‘0’ and the patterns are treated as similar (representing a stillimage).

If the block matching quantity blkpm is between the offset value M1 andthe given value M2, the block coefficient dmb11 increases from ‘0’ toMax_k as the block matching quantity blkpm increases, indicatingincreasing degrees of motion (indicating a state between the statestreated as definitely moving and definitely still, and showing how closethe state is to the definitely moving state).

When the magnification factor Mk1 is increased, the block matchingquantity blkpm is converted to a block coefficient dmb11 with largervalues, indicating greater motion, so that the motion becomes moreeasily recognizable. Increasing the offset value M1 increases the valueup to which difference values are detected as still parts of the image.The motion detection sensitivity can accordingly be adjusted in theconversion process that yields the block coefficient dmb11 by adjustingthe magnification factor Mk1 and offset value M1.

The block coefficient converter 302 a may generate the block coefficientdmb11 by comparing the block matching quantity blkpm with a giventhreshold, or by using a lookup table (LUT) in a read-only memory (ROM)to perform a predetermined conversion such as, for example, theconversion shown in FIG. 9, instead of by carrying out the nonlinearconversion as described above.

The matching vertical high frequency component vebpm from the matchingvertical high frequency component extractor 304 is converted to a highfrequency component coefficient hldm1, which indicates the degree ofmotion in the vertical motion component in the pixel block, bycomparison with a threshold or by a nonlinear conversion in the same wayas in the block coefficient converter 302 a, and is output to the highfrequency component converter 305 a in the first moving blockcoefficient conversion unit 311.

The high frequency component coefficient hldm1 increases from ‘0’ to ‘1’as the matching vertical high frequency component vebpm increases,indicating increasing degrees of motion in the vertical motioncomponent.

The coefficient combiner 314 receives the block coefficient dmb11 fromthe block coefficient converter 302 a and the high frequency componentcoefficient hldm1 from the high frequency component converter 305 a. Thecoefficient combiner 314 combines the block coefficient dmb11 and thehigh frequency component coefficient hldm1 and generates and outputs themoving block coefficient dfmat. The coefficient combiner 314 combinesthe values, for example, by converting the block coefficient dmb11 inaccordance with the value of the high frequency component coefficienthldm1.

Since the high frequency component coefficient hldm1 indicates thedegree of motion in the vertical motion component, if the coefficientcombiner 314 converts the block coefficient dmb11 by multiplying it bythe high frequency component coefficient hldm1, for example, the blockcoefficient dmb11 is output directly as the moving block coefficientdfmat when the high frequency component coefficient hldm1 is ‘1’(indicating that a vertical motion component is present). When the highfrequency component coefficient hldm1 is ‘0’ (indicating that novertical motion component is present), the moving block coefficientdfmat is output as ‘0’. More generally, the block coefficient dmb11 isweighted in accordance with the value of the high frequency componentcoefficient hldm1 to obtain the moving block coefficient dfmat.

The moving block coefficient dfmat output from the coefficient combiner314 can thus be obtained as a moving block coefficient dfmat thatindicates the degree of block motion by reflecting vertical motion ofpatterns in the pixel block.

The coefficient combiner 314 can generate the moving block coefficientdfmat reflecting vertical motion of the pixel block pattern byoutputting the block coefficient dmb11 as dfmat when the high frequencycomponent coefficient hldm1 exceeds a given value (for example whenhldm1=1) instead of by multiplying the block coefficient dmb11 by thehigh frequency component coefficient hldm1.

In the description given above the moving block coefficient dfmatdetermined by the coefficient combiner 314 takes account of verticalpattern motion, but the block coefficient dmb11 output from the blockcoefficient converter 302 a may be output directly as moving blockcoefficient dfmat. It is sufficient if moving block coefficient dfmat isgenerated from the block matching quantity blkpm and indicates motion ofthe pixel block. Any moving block coefficient dfmat generated in thisway indicates that the pixel blocks have different patterns, that is,that the pixel blocks are moving.

When a conversion is made in accordance with the high frequencycomponent coefficient hldm1 in order to reflect vertical pattern motion,moving block coefficient dfmat enables appropriate processing to becarried out in accordance with pattern motion in the frame differencedetection process and other processes described later.

The moving block coefficient dfmat generated by the coefficient combiner314 in the first moving block coefficient conversion unit 311 is sent tothe frame difference detector 20 in FIG. 1 as the moving blockcoefficient dfmat detected by the pattern matching detector 30.

Since the second and third moving block coefficient conversion units312, 313 are similar in configuration to the first moving blockcoefficient conversion unit 311, differing only in their inputs,detailed descriptions will be omitted. The second moving blockcoefficient conversion unit 312 receives the block matching quantityblkpm and the matching vertical high frequency component vebpm extractedby the matching vertical high frequency component extractor 304 anddetermines a horizontal line moving block coefficient dfmat_mkreflecting vertical pattern motion. The third moving block coefficientconverter 313 receives the block matching quantity blkpm and thematching horizontal high frequency component hebpm extracted by thematching horizontal high frequency component extractor 307 anddetermines a vertical line moving block coefficient Vrdfmat reflectinghorizontal pattern motion. The horizontal line moving block coefficientdfmat_mk and the vertical line moving block coefficient Vrdfmat are sentto the motion information corrector 50 in FIG. 1 as the moving blockcoefficients cbmat detected by the pattern matching detector 30.

The block coefficient converters 302 b, 302 c, high frequency componentconverters 305 b, 305 c, and coefficient combiners 315, 316 in thesecond and third moving block coefficient conversion units 312, 313 havethe same respective configurations as the block coefficient converter302 a, high frequency component converter 305 b, and coefficientcombiner 314 in the first moving block coefficient conversion unit 311.The moving block coefficients cbmat they obtain can take vertical orhorizontal pattern motion into account indicating block pattern motionin the pixel block responsive to the value of the block matchingquantity blkpm, based on pattern similarity.

The first, second, and third moving block coefficient conversion units311, 312, 313 may perform a common conversion to obtain a singlecoefficient instead of performing separate conversions to obtainseparate moving block coefficients. However, by specifying differentmagnification factors and offset values for separate conversionsperformed by the block coefficient converters 302 a, 302 b, 302 c andhigh frequency component converters 305 a, 305 b, 305 c, the highfrequency component direction and detection sensitivity can be adjustedin the conversion process to obtain moving block coefficients for moreappropriate processing responsive to pattern motion during framedifference detection and in the processing in the motion informationcorrector 50, which will be described later.

As described above, the pattern matching detector 30 in FIG. 2A obtainsa block matching quantity blkpm based on a calculation of similaritybetween the pixel block of interest b0 in the current frame Di0 and thereference pixel block b2 in the immediately preceding frame d2 f,obtains vertical and horizontal high frequency components of theabsolute difference, and generates moving block coefficients dfmat,dbmat responsive to similarity based on the block matching quantity,reflecting whether the pixel block has vertical or horizontal patternmotion. Since the moving block coefficients dfmat, dbmat are obtainedfrom pattern matching (similarity calculation) on the pixel blocks, avalue indicating block pattern motion in the pixel blocks can beobtained because the difference in pattern between the pixel blocks isconsidered in addition to the difference between the pixel of interestP0 and the corresponding pixel.

Referring again to FIG. 1, the edge detector 40 receives the currentframe signal Di0, one-frame delayed signal d2 f, and one-field delayedsignal d1 f, detects edges in the image in the vicinity of the pixel ofinterest P0 in the current frame Di0 or in the vicinity of the positioncorresponding to the pixel of interest P0 in the immediately precedingfield or frame from the pixel values of the pixels represented by thecurrent frame signal Di0, one-frame delayed signal d2 f, and one-fielddelayed signal d1 f, and outputs edge decision coefficients efkm, cedkm.

The edge detector 40 is configured as shown, for example, in FIG. 10.

The edge detector 40 in FIG. 10 includes a vertical edge detectionsection 41, which detects vertical high frequency components (verticaledge components, or horizontal line components) in the image and ahorizontal edge detection section 42, which detects horizontal highfrequency components (horizontal edge components, or vertical linecomponents) in the image.

The vertical edge detection section 41 and horizontal edge detectionsection 42 detect edges by extracting vertical and horizontal highfrequency components from the current frame signal Di0, the signal d2 fof the frame one frame before, and the signal dif of the frame one fieldbefore, specifically from the pixel of interest P0 and pixels in thevicinity of the pixel of interest P0 in the same frame and thecorresponding pixels in the temporally adjacent frames, and outputs theresults as edge decision coefficients.

The vertical edge detection section 41 includes vertical high frequencycomponent extractors 411, 412, 413, that extract vertical edgecomponents, which are vertical high frequency components, a verticalhigh frequency component selector 414, vertical edge decisioncoefficient converters 415, 416, 417, and an inverter 418.

The vertical high frequency component extractors 411, 412, 413 havesimilar configurations but receive signals from different frames orfields. The vertical edge decision coefficient converters 415, 416, 417have similar configurations but can operate with different coefficientconversion settings, which can be set and modified separately(independently).

The horizontal edge detection section 42 includes horizontal highfrequency component extractors 421, 422, 423, that extract horizontaledge components, which are horizontal high frequency components, ahorizontal high frequency component selector 424, and horizontal edgedecision coefficient converters 425, 426.

The horizontal high frequency component extractors 421, 422, 423 havesimilar configurations but receive signals from different frames orfields. The horizontal edge decision coefficient converters 425, 426have similar configurations but can operate with different coefficientconversion settings, which can be set and modified separately(independently).

As in the description of the pattern matching detector 30, FIG. 5illustrates temporal relationships among the current frame signal Di0,one-frame delayed signal d2 f, and one-field delayed signal d1 f andpositional relationships in the vertical scanning direction among pixelsused in the processing, when the video signal processing device receivesan interlaced video signal.

In FIG. 5, on the vertical line representing the current frame Di0, thepixel of interest P0 (central pixel of the pixel block b0 in FIG. 6B) isshown on line n. On the vertical line representing the frame of theone-frame delayed signal d2 f, the pixel Pf2 (central pixel of the pixelblock b2 in FIG. 6A) is also shown on line n (in the same verticalposition as the pixel of interest). On the vertical line representingthe frame of the one-field delayed signal d1 f, pixel Pfla is shown halfa line above the pixel of interest, and pixel Pflb half a line below.

The operation of the edge detector 40 will next be described withreference to FIGS. 5 and 10.

In the vertical edge detection section 41, the current frame signal Di0,one-frame delayed signal d2 f, and one-field delayed signal d1 f areinput to the vertical high frequency component extractors 411, 412, 413,respectively. The vertical high frequency component extractors 411, 412,413 extract vertical high frequency components, which are vertical edgecomponents (horizontal line components), in the vicinity of the pixel ofinterest P0 (or a corresponding position), from the current frame signalDi0, one-frame delayed signal d2 f, one-field delayed signal d1 f.

The vertical edge components are extracted by bandpass filters (BPFs)and the like. After vertical BPF processing of pixels of differentlines, the absolute values of the outputs of the vertical BPFs areobtained as vertical edge components.

The absolute values of the BPF outputs may be subjected to a smoothingprocess by a horizontal LPF process. Alternatively, prior to thevertical BPF process, a horizontal LPF process may be carried out on thesignals on each line. The essential point is to obtain vertical edgecomponents of contours extending horizontally or having a horizontalcomponent.

The vertical high frequency component extractors 411 and 412 havesimilar configurations. The vertical high frequency component extractor411 receives signals representing pixel P0 and pixels (such as P0 a, P0b in FIG. 5) on the same vertical line in the current frame Di0 througha line delay process or the like, extracts a vertical edge component(the absolute value of a vertical BPF output, for example) fromline-to-line pixel calculations (calculations using values of pixels ondifferent lines) in the same frame, and outputs the result as a verticaledge component vbp0.

The vertical high frequency component extractor 412 receives signalsrepresenting pixel Pf2 (in the same position as the pixel P0) and pixels(Pf2 a, Pf2 b in FIG. 5) on the same vertical line in the frame of theone-frame delayed signal d2 f, extracts a vertical edge component fromline-to-line pixel calculations (calculations using values of pixels ondifferent lines) in that frame, and outputs it as a vertical edgecomponent vbp2.

Vertical edge components may also be extracted by calculations involvingonly the pixel in position P0 or Pf2 and a single pixel disposed on theline above or below, or by calculations on pixels disposed on a range ofseveral lines extending above and below pixel position P0 or Pf2.

The extracted vertical edge component may be subjected to a smoothingprocess, gain adjustment, or other conversion process.

The vertical high frequency component extractor 413 receives signals ofpixels (for example, Pf1 a and Pf1 b in FIG. 5) on a vertical line ofthe one-field delayed signal d1 f by a line delay process or the like,extracts a vertical edge component (the absolute value of a vertical BPFoutput based on the difference between two pixels, for example) fromline-to-line pixel calculations (calculations using values of pixels ondifferent lines) in the field, and outputs it as the vertical edgecomponent vbp1 f of the field.

The vertical edge component may also be extracted from calculations onpixels on a range of several lines above and below the pixels Pf1 a andPf1 b in FIG. 5. The same number of lines may be used as in the verticalhigh frequency component extractor 411. The extracted vertical edgecomponent may be subjected a smoothing process, gain adjustment, orother conversion process, also as described above.

The vertical edge components vbp0, vbp2, vbp1 f in the frames of thecurrent frame signal Di0, one-frame delayed signal d2 f, and one-fielddelayed signal d1 f are output from the vertical high frequencycomponent extractors 411, 412, 413 to the vertical high frequencycomponent selector 414.

The vertical high frequency component selector 414 selects the minimumvalue of the input vertical edge components vbp0, vbp2, vbp1 f andoutputs it as an extracted vertical edge component ved.

The extracted vertical edge component ved is the minimum value of thevertical high frequency components obtained in the vicinity of the pixelof interest P0 (the pixels around the pixel of interest P0 in FIG. 5)and the vicinity of the corresponding positions in the period up to theframe one frame before. Since the minimum value is selected, if thevalue of the extracted vertical edge component is greater than athreshold specified for determining the presence of a vertical highfrequency component, it can be determined that the current frame signalDi0, the one-frame delayed signal d2 f, and the one-field delayed signaldif each have a vertical high frequency component, or a horizontal linecomponent or vertical edge.

The vertical high frequency component selector 414 may alternativelyselect the maximum value of the vertical edge components as theextracted vertical edge component ved. In that case, whether ahorizontal line component, that is, a vertical edge, is present in anyone of the current frame signal Di0, the one-frame delayed signal d2 f,and the one-field delayed signal d1 f is determined.

The extracted vertical edge component ved indicating the presence of ahorizontal line component or vertical edge in the vicinity of the pixelof interest P0 is output from the vertical high frequency componentselector 414 to the vertical edge decision coefficient converters 415,416, 417.

The vertical edge decision coefficient converters 415, 416, 417 decidefrom the extracted vertical edge component ved whether there is avertical edge and generate vertical edge decision coefficients. Thevertical edge decision coefficient converters 415, 416, 417 have similarconfigurations but can have independent coefficient conversion settingsfor generating different vertical edge decision coefficients.

Since the vertical edge decision coefficient converters 415, 416, 417are similar in configuration, only vertical edge decision coefficientconverter 415 will be described next.

For example, the vertical edge decision coefficient converter 415subtracts an offset value E1 from the extracted vertical edge componentved, multiplies the difference by a magnification factor Ek1, performs anonlinear conversion by limiting the product ((ved−E1)×Ek1) to a givenrange (e.g., 0 to Max_ek), and outputs the result as a first verticaledge decision coefficient vedg_fk.

Alternatively, the extracted vertical edge component ved may bemultiplied by the magnification factor Ek1 before subtracting the offsetvalue E1 (ved×Ek1−E1).

FIG. 11 shows a conversion curve by which the vertical edge decisioncoefficient converter 415 converts the extracted vertical edge componentved (the extracted edge component in the drawing) to the vertical edgedecision coefficient vedg_fk (the edge decision coefficient in thedrawing). The illustrated vertical edge decision coefficient vedg_fkranges from 0 to the maximum value Max_ek.

The vertical edge decision coefficient vedg_fk is used as an indicatorof the degree of presence of a horizontal line component or verticaledge (degree to which the block is to be treated as having a horizontalline component or vertical edge).

The values of the vertical edge decision coefficient vedg_fk from ‘0’ to‘1’ may be converted to values from ‘0’ to ‘8’ representing fractions1/8, 2/8, and so on, with ‘8’ representing ‘1’.

In the example shown, when the extracted vertical edge component ved isequal to or greater than a given value E2, the vertical edge decisioncoefficient vedg_fk takes the maximum value Max_ek, indicating the(definite) presence of a vertical edge. When the extracted vertical edgecomponent ved is equal to or less than the offset value E1, the verticaledge decision coefficient vedg_fk is ‘0’, indicating the (definite)absence of a vertical edge.

If the extracted vertical edge component ved is between the offset valueE1 and the given value E2, the vertical edge decision coefficientvedg_fk increases from 0 to Max_ex as the extracted vertical edgecomponent ved increases, indicating increasing degrees to which thecomponent should be treated as a vertical edge, or in other words,indicating the size or strength of the edge component (indicating astate between the states treated as a definite edge and definitely noedge, and showing how close the state is to the definite edge state).

When the magnification factor Ek1 is increased, the extracted verticaledge component ved is converted to a vertical edge decision coefficientvedg_fk with larger values, making it easier to detect the presence of avertical edge. Increasing the offset value E1 expands the range overwhich the absence of an edge is detected. High-frequency componentswithin that range are treated as noise. The edge detection sensitivitycan accordingly be adjusted in the conversion process that yields thevertical edge decision coefficient vedg_fk by adjusting themagnification factor Ek1 and the offset value E1.

The vertical edge decision coefficient converter 415 may generate thevertical edge decision coefficient vedg_fk by comparing the extractedvertical edge component ved with a given threshold or by using a LUT toperform a predetermined conversion such as, for example, the conversionshown in FIG. 11, instead of by carrying out the nonlinear conversion asdescribed above.

The vertical edge decision coefficient converters 416, 417 are similarin configuration to the vertical edge decision coefficient converter415, so that detailed descriptions will be omitted. The extractedvertical edge component ved is converted to a second vertical edgedecision coefficient vedg_mk by vertical edge decision coefficientconverter 416 and to a third vertical edge decision coefficient vrde byvertical edge decision coefficient converter 417. The vertical edgedecision coefficient vrde obtained by vertical edge decision coefficientconverter 417 is sent to the inverter 418.

The vertical edge decision coefficient converters 416, 417 have the sameconfiguration as the vertical edge decision coefficient converters 415.The vertical edge decision coefficients vedg_mk, vrde obtained from thevertical edge decision coefficient converters 416, 417 both take valuesfrom ‘0’ to ‘1’, indicating the degree of presence of a horizontal linecomponent or vertical edge (degree to which the block is to be treatedas having a horizontal line component or vertical edge), responsive tothe value of the extracted vertical edge component ved.

The vertical edge decision coefficient converters 415, 416, 417 mayperform a common conversion to obtain a single coefficient instead ofperforming separate conversions to obtain separate vertical edgedecision coefficients. However, by specifying different magnificationfactors and offset values for separate conversions by the vertical edgedecision coefficient converters 415, 416, 417, the edge detectionsensitivity can be adjusted in the conversion process that yields thevertical edge decision coefficients for more appropriate processingresponsive to edge direction during frame difference detection and inthe processing in the motion information corrector 50, which will bedescribed later.

The inverter 418 inverts the vertical edge decision coefficient vrdefrom the vertical edge decision coefficient converter 417 and outputs aninverted vertical edge decision coefficient vrd_hvkm. The inversion iscarried out by, for example, subtracting the value of the vertical edgedecision coefficient vrde from the maximum value. The vertical edgedecision coefficient vrde indicates the degree of presence of ahorizontal line component or vertical edge; the inverted vertical edgedecision coefficient indicates the degree of absence of a horizontalline component or vertical edge, or the degree of non-verticality of theedge.

The vertical edge decision coefficients vedg_fk and vedg_mk generated byvertical edge decision coefficient converters 415 and 416 and thenon-vertical edge decision coefficient vrd_hvkm generated by inverter418 indicate whether there is a vertical edge in the vicinity of thepixel of interest P0.

In the horizontal edge detection section 42, the current frame signalDi0, one-frame delayed signal d2 f, and one-field delayed signal d1 fare input to the horizontal high frequency component extractors 421,422, 423, respectively. The horizontal high frequency componentextractors 421, 422, 423 extract horizontal high frequency components,which are horizontal edge components (vertical line components), in thevicinity of the pixel of interest P0 (or a corresponding position), fromthe current frame signal Di0, one-frame delayed signal d2 f, andone-field delayed signal d1 f.

In the horizontal edge component extraction, a vertical LPF extracts avertical low frequency component, then a horizontal BPF process isperformed, and the absolute value of the output of the horizontal BPF isobtained as a horizontal edge component.

The horizontal edge component extraction may be carried out by ahorizontal BPF process centered on a given pixel. A smoothing processmay be performed by performing a vertical LPF process on the absolutevalue of the horizontal BPF output. The essential point is to obtainhorizontal edge components of contours extending vertically or having avertical component.

The horizontal high frequency component extractors 421 and 422 havesimilar configurations. The horizontal high frequency componentextractor 421 operates on pixel P0 in the current frame Di0 byextracting the vertical low frequency component with a vertical LPF andthen performing a horizontal BPF process on the vertical low frequencycomponent to extract a horizontal edge component (for example, theabsolute value of the horizontal BPF output), and outputs the result asa horizontal edge component hbp0.

The horizontal high frequency component extractor 422 operates on pixelPf2 in the frame of the one-frame delayed signal df2 by extracting avertical low frequency component with a vertical LPF and then performinga horizontal BPF process on the vertical low frequency component toextract a horizontal edge component, which is output as a horizontaledge component hbp2.

The extracted horizontal edge component may be subjected to a smoothingprocess, gain adjustment, or other conversion process.

The horizontal high frequency component extractor 423 operates on pixelsPf1 a, Pf1 b (see FIG. 5) of the one-field delayed signal d1 f byextracting a vertical low frequency component (for example, the meanvalue of the two pixels) by a vertical LPF process and performing ahorizontal BPF process on the vertical low frequency component toextract a horizontal edge component (for example, the absolute value ofthe BPF output) from the vertical low frequency component in the field,and outputs the result as the horizontal edge component hbp1 f in thefield.

The vertical low frequency component may also be extracted by a verticalLPF process carried out on pixels disposed on a range of several linesabove and below the pixels Pf1 a, Pf1 b in FIG. 5. The extractedhorizontal edge component may be subjected to a smoothing process, gainadjustment, or other conversion process, as with the above.

The horizontal edge components hbp0, hbp2, hbp1 f in the frames of thecurrent frame signal Di0, one-frame delayed signal d2 f, and one-fielddelayed signal d1 f are output from the horizontal high frequencycomponent extractors 421, 422, 423 to the horizontal high frequencycomponent selector 424.

The horizontal high frequency component selector 424 selects the minimumvalue of the input horizontal edge components hbp0, hbp2, hbp1 f andoutputs it as an extracted horizontal edge component hed.

The extracted horizontal edge component hed is the minimum value of thehorizontal high frequency components obtained in the vicinity of thepixel of interest P0 (the pixels around the pixel of interest P0 in FIG.5) and the vicinity of the corresponding positions in the period up tothe frame one frame before. Since the minimum value is selected, if thevalue of the extracted horizontal edge component hed is greater than athreshold specified for determining the presence of a horizontal highfrequency component, it can be determined that the current frame signalDi0, the one-frame delayed signal d2 f, and the one-field delayed signald1 f each have a horizontal high frequency component, representing avertical line or horizontal edge.

The horizontal high frequency component selector 424 may alternativelyselect the maximum value of the horizontal edge components as theextracted horizontal edge component hed. In that case, whether avertical line component, that is, a horizontal edge, is present in anyone of the current frame signal Di0, the one-frame delayed signal d2 f,and the one-field delayed signal d1 f is determined.

The extracted horizontal edge component hed indicating the presence of avertical line component or horizontal edge in the vicinity of the pixelof interest P0 is output from the horizontal high frequency componentselector 424 to the horizontal edge decision coefficient converters 425,426.

The horizontal edge decision coefficient converters 425, 426 decide fromthe extracted horizontal edge component hed whether there is ahorizontal edge and generate horizontal edge decision coefficients. Thehorizontal edge decision coefficient converters 425, 426 have similarconfigurations but can have independent coefficient conversion settingsfor generating different horizontal edge decision coefficients.

Since the horizontal edge decision coefficient converters 425, 426 aresimilar in configuration, only horizontal edge decision coefficientconverter 425 will be described next.

The horizontal edge decision coefficient converter 425 converts theextracted horizontal edge component hed to a first horizontal edgedecision coefficient hedg_fk. The conversion is effected by, forexample, a process similar to the process used in the vertical edgedecision coefficient converter 415. FIG. 11 shows the conversion curve,which ranges from ‘0’ to a maximum value Max_ek.

The horizontal edge decision coefficient hedg_fk is used as an indicatorof the degree of presence of a vertical line component or horizontaledge.

As in the conversion performed in the vertical edge decision coefficientconverter 415, the values of the horizontal edge decision coefficienthedg_fk from ‘0’ to ‘1’ may be converted to values from ‘0’ to ‘8’representing fractions ⅛, 2/8, and so on, with ‘8’ representing ‘1’.

As with the extracted vertical edge component ved, if the extractedhorizontal edge component hed is equal to or greater than apredetermined value E2, the horizontal edge decision coefficient hedg_fktakes the maximum value Max_ek, indicating the (definite) presence of ahorizontal edge. If the extracted horizontal edge component hed is equalto or less than an offset value E1, the horizontal edge decisioncoefficient hedg_fk is ‘0’, indicating the (definite) absence of anedge.

If the extracted horizontal edge component hed is between the offsetvalue E1 and the given value E1, the horizontal edge decisioncoefficient hedg_fk increases from ‘0’ to Max_ek as the extractedhorizontal edge component hed increases, indicating increasing degreesto which the value represents a horizontal edge, or should be treated asrepresenting a horizontal edge.

The horizontal edge decision coefficient converter 426 has the sameconfiguration as the horizontal edge decision coefficient converter 425,so that a description will be omitted. The horizontal edge decisioncoefficient converter 426 converts the extracted horizontal edgecomponent hed and generates a second horizontal edge decisioncoefficient hedg_mk. The horizontal edge decision coefficient hedg_mktakes values from ‘0’ to ‘1’, indicating the degree of the presence of avertical line component or horizontal edge, responsive to the value ofthe extracted horizontal edge component hed.

The horizontal edge decision coefficient converters 425, 426 may performa common conversion to obtain a single coefficient instead of performingseparate conversions to obtain separate horizontal edge decisioncoefficients. However, by specifying different magnification factors andoffset values for the separate conversions performed by the horizontaledge decision coefficient converters 425, 426, the edge detectionsensitivity can be adjusted in the conversion process that yields thehorizontal edge decision coefficients for more appropriate processingresponsive to edge direction in the image during frame differencedetection and in the processing in the motion information corrector 50,which will be described later.

The horizontal edge decision coefficients hedg_fk and hedg_mk generatedby the horizontal edge decision coefficient converters 425, 426 indicatewhether there is a horizontal edge in the vicinity of the pixel ofinterest P0.

As has been described, the edge detector 40 shown in FIG. 10 extractshigh frequency components of the signals from the pixels in the currentframe signal Di0, one-frame delayed signal d2 f, and one-field delayedsignal d1 f, and generates and outputs edge decision coefficients efkmand cedkm indicating the degree of presence of an edge component in thevicinity of the pixel of interest P0.

The vertical edge decision coefficient vedg_fk from the vertical edgedecision coefficient converter 415 in the vertical edge detectionsection 41 and the horizontal edge decision coefficient hedg_fk from thehorizontal edge decision coefficient converter 425 in the horizontaledge detection section 42 are sent to the frame difference detector 20in FIG. 1 as the edge decision coefficients efkm indicating the presenceof an edge in the vicinity of the pixel of interest P0, as detected bythe edge detector 40. The vertical edge decision coefficient vedg_mkfrom the vertical edge decision coefficient converter 416 in thevertical edge detection section 41, the non-vertical edge decisioncoefficient vrd_hvkm from the inverter 418, and the horizontal edgedecision coefficient hedg_mk from the horizontal edge decisioncoefficient converter 426 in the horizontal edge detection section 42are sent to the motion information corrector 50 in FIG. 1 as the edgedecision coefficients cedkm detected by the edge detector 40.

Referring again to FIG. 1, the frame difference detector 20 receives thecurrent frame signal Di0, the one-frame delayed signal d2 f, and alsothe moving block coefficient dfmat from the pattern matching detector 30and the edge decision coefficients efkm from the edge detector 40.

The frame difference detector 20 obtains the difference between in-phasepixels in the preceding frame signal d2 f and the current frame signalDi0, selects the smoothing extent according to the moving blockcoefficient dfmat and edge decision coefficients efkm corresponding tothe difference, performs a smoothing process within the selected extent,detects a smoothed frame-to-frame difference, and generates a framedifference signal fdiff as a detection result.

The frame difference signal fdiff output from the frame differencedetector 20 is ‘0’ in still parts of the image, and its absolute valueincreases as the degree of motion increases.

The frame difference calculator 21 in the frame difference detector 20receives the current frame signal Di0 and the one-frame delayed signald2 f and also receives the moving block coefficient dfmat from thepattern matching detector 30 and the edge decision coefficients efkmfrom the edge detector 40.

The frame difference calculator 21 is configured as shown, for example,in FIG. 12.

The frame difference calculator 21 in FIG. 12 includes a subtractor 211,a horizontal smoothing processor 212, a first mixer 213, a verticalsmoothing processor 214, a second mixer 215, and an absolute valuecalculator 216.

The moving block coefficient dfmat input from the pattern matchingdetector 30 to the frame difference calculator 21 indicates a degree ofmotion reflecting vertical pattern motion in the pixel block. The edgedecision coefficients efkm input from the edge detector 40 are thevertical edge decision coefficient vedgfk indicating the degree ofvertical edge component (horizontal line) and the horizontal edgedecision coefficient hedgfk indicating the degree of horizontal edgecomponent (vertical line).

The subtractor 211 in the frame difference calculator 21 in FIG. 12receives the current frame signal Di0 and the one-frame delayed signald2 f. The subtractor 211 calculates differences between in-phase pixelsin the one-frame delayed signal d2 f and the current frame signal Di0.The difference Dn obtained for the pixel of interest is the differencein pixel value of the pixel of interest in the current frame signal Di0and the pixel in the matching position in the one-frame delayed signald2 f.

The horizontal smoothing processor 212 receives the difference Dnobtained by the subtractor 211 and performs a smoothing process on ahorizontal pixel extent. That is, the difference Dn at the position ofthe pixel of interest P0 is averaged with the differences Dn within anextent of horizontally adjacent pixel positions centered on the pixel ofinterest P0, and the resulting mean value is output as a horizontallysmoothed difference hlpd obtained by performing a smoothing process onthe given horizontal pixel extent. In the averaging process, a weightedaverage may be taken instead of taking a simple average of differencesin the horizontal direction.

For example, the smoothing process may be performed as a horizontal LPFprocess that takes a weighted average in which the weights increase asthe pixel position approaches the center position.

When the difference Dn is smoothed within a given horizontal extent inthe horizontal smoothing processor 212, if there is a moving horizontaledge such as a moving vertical line, since the differences Dn of pixelsnear the horizontal edge are averaged, any abrupt change disappears (alow frequency component is obtained), and the smoothed values becomesmall despite the presence of motion.

In particular, if there is a moving object with a repetitive pattern ofvertical lines, positive differences alternate with negativedifferences, so that the smoothed values become quite small, despite thepresence of motion.

In flat parts of the image or parts without horizontal edge components,however, the smoothing process can reduce the effect of high frequencynoise components on the differences.

The first mixer 213 obtains a horizontally mixed difference hdf bymixing the difference Dn from the subtractor 211 and the horizontallysmoothed difference hlpd from the horizontal smoothing processor 212 inaccordance with the first horizontal edge decision coefficient hedg_fkin the first edge decision coefficients efkm from the edge detector 40.

This horizontal edge decision coefficient hedg_fk depends on the degreeof presence of a horizontal edge component (vertical line). The firstmixer 213 mixes the difference Dn and the horizontally smootheddifference hlpd in accordance with the horizontal edge decisioncoefficient hedg_fk, as in the following equation (26), for example.

hdf=(1−hedg _(—) fk)×hlpd+hedg _(—) fk×Dn  (26)

If the mixing is carried out as in equation (26) and the value of thehorizontal edge decision coefficient ranges from ‘0’ to ‘1’, ahorizontally mixed difference hdf obtained by mixing the difference Dnand the horizontally smoothed difference hlpd in a mixing ratioresponsive to the value of the horizontal edge decision coefficienthedg_fk is output. When the horizontal edge decision coefficient hedg_fkis ‘1’, indicating the presence of a horizontal edge component, thedifference Dn itself is output as the horizontally mixed difference hdf.If the horizontal edge decision coefficient hedg_fk is ‘0’, thehorizontally smoothed difference hlpd is output as the horizontallymixed difference hdf.

A difference that has been smoothed within an appropriate extent inaccordance with the value of the horizontal edge decision coefficienthedg_fk is thereby obtained.

The first mixer 213 may select the difference Dn when the horizontaledge decision coefficient hedg_fk is equal to or greater than a givenvalue (e.g., hedg_fk=1, for example) and may select the horizontallysmoothed difference hlpd otherwise. This type of selection process canbe regarded as a mixing process in which the mixing ratio is either ‘1’or ‘0’.

The moving block coefficient dfmat from the pattern matching detector 30could be used to have the first mixer 213 select the difference Dn whenthere is a moving pattern and the horizontal edge decision coefficienthedg_fk indicates the presence of a horizontal edge component. Whenthere is a horizontal edge component (vertical line), however, failureto detect the difference as motion could lead to a failure to detectmotion. It is therefore better to obtain the horizontally mixeddifference hdf in accordance with the horizontal edge decisioncoefficient hedg_fk, regardless of whether there is pattern motion, sothat a difference that can be detected as motion can be obtained.

The vertical smoothing processor 214 receives the horizontally mixeddifference hdf from the first mixer 213 and performs a smoothing processover a vertical pixel extent. That is, the horizontally mixed differencehdf at the pixel of interest P0 is averaged with the horizontally mixeddifferences hdf within an extent of vertically adjacent pixel positionscentered on the pixel of interest P0, and the resulting mean value isoutput as a vertically smoothed difference vlpd obtained by performing asmoothing process on the given vertical pixel extent. In the averagingprocess, a weighted average may be taken instead of taking a simpleaverage of differences in the vertical direction. For example, thesmoothing process may be performed as a vertical LPF process that takesa weighted average in which the weights increase as the pixel positionapproaches the center position.

When the horizontally mixed difference hdf is smoothed within a givenvertical extent in the vertical smoothing processor 214, if there is amoving vertical edge such as a moving horizontal line, since thedifferences hdf of pixels near the vertical edge are averaged, anyabrupt change disappears (a low frequency component is obtained), andthe smoothed values become small despite the presence of motion.

In particular, if there is a moving object with a repetitive pattern ofhorizontal lines, positive differences alternate with negativedifferences, so that the smoothed values become quite small, despite thepresence of motion.

In flat parts of the image or parts without vertical edge components,however, the smoothing process can reduce the effect of high frequencynoise components on the differences.

Even when a vertical edge component such as a horizontal line ispresent, if the edge is stationary or moves in the horizontal direction,the reduction in difference value due to the vertical smoothing processwill not lead to a failure to detect motion.

The second mixer 215 obtains a mixed difference dif by mixing thehorizontally mixed difference hdf from the mixer 213 and the verticallysmoothed difference vlpd from the vertical smoothing processor 214 inaccordance with the first moving block coefficient dfmat output from thepattern matching detector 30 and the first vertical edge decisioncoefficient vedg_fk in the first edge decision coefficients efkm outputfrom the edge detector 40.

The moving block coefficient dfmat reflects the degree of block motionreflecting vertical pattern motion in the pixel block, and the verticaledge decision coefficient vedg_fk reflects the degree of presence of avertical edge component (horizontal line). The mixer 215 operates by,for example, multiplying the vertical edge decision coefficient vedg_fkby the moving block coefficient dfmat to obtain a mixing ratio vdkranging from ‘0’ to ‘1’ (vdk=vedg_fk×dfmat), and then mixing thehorizontally mixed difference hdf and the vertically smoothed differencevlpd according to the mixing ratio vdk, as in the following equation(27).

dif=(1−vdk)×vlpd+vdk×hdf  (27)

If the moving block coefficient dfmat is ‘1’, indicating verticalpattern motion, and the vertical edge decision coefficient vedg_fk is‘1’, indicating the presence of a vertical edge component, the movingpattern is a horizontal line component or vertical edge and the mixingratio vdk is ‘1’.

When the value of the mixing ratio vdk is between ‘0’ and ‘1’, the mixeddifference dif is obtained by mixing the horizontally mixed differencehdf and the vertically smoothed difference vlpd in accordance with themixing ratio vdk. When the mixing ratio vdk is ‘1’, the horizontallymixed difference hdf itself is output as the mixed difference dif. Whenthe mixing ratio vdk is ‘0’, the vertically smoothed difference vlpd isoutput as the mixed difference dif.

A difference that has been smoothed within an appropriate extent inaccordance with the moving block coefficient dfmat and the vertical edgedecision coefficient vedg_fk is thereby obtained.

The mixer 215 may select the horizontally mixed difference hdf when boththe moving block coefficient dfmat and the vertical edge decisioncoefficient vedg_fk exceed a given value (such as ‘1’) and select thevertically smoothed difference vlpd otherwise. This type of selectionprocess can be regarded as a mixing process in which the mixing ratio iseither ‘1’ or ‘0’. With the mixing process expressed by equation (27),the selection of the horizontally mixed difference hdf and thevertically smoothed difference vlpd can be switched smoothly, and moreaccurate differences can be obtained.

The absolute value calculator 216 receives the mixed difference dif fromthe second mixer 215, calculates the absolute value of the mixeddifference dif, and outputs it as the frame difference frd.

The absolute value of the frame difference obtained by the absolutevalue calculator 216 is output as the frame difference frd from theframe difference calculator 21. When the horizontal edge decisioncoefficient hedg_fk indicates the presence of a horizontal edgecomponent (vertical line), the frame difference frd is obtained after avertical smoothing process, without horizontal smoothing. When themoving block coefficient dfmat indicates vertical pattern motion and thevertical edge decision coefficient vedg_fk indicates the presence of avertical edge component (horizontal line), the frame difference isobtained after a horizontal smoothing process, without verticalsmoothing. In other cases, the frame difference is obtained after bothhorizontal and vertical smoothing processes. The effect of switching theextent of the smoothing process in the frame difference calculator 21 inaccordance with edge direction and pattern motion in the image is thataccurate frame differences, smoothed within appropriate extents, can beobtained, avoiding the mistaken detection of a still part as moving dueto noise, or the mistaken detection of a moving part as still due tofailure to detect motion.

Since the extent of the smoothing process in the frame differencecalculator 21 is switched not according to a threshold but according toa mixing ratio determined from the moving block coefficient dfmat andedge decision coefficients efkm, a smooth switchover can be made betweenselecting the horizontally mixed difference hdf and selecting thevertically smoothed difference vlpd, making it possible to obtain anaccurate, stable frame difference.

In the frame difference calculator 21 in FIG. 12, the frame differencefrd is obtained from the vertical smoothing processor 214 and secondmixer 215, which are disposed in a stage succeeding the horizontalsmoothing processor 212 and first mixer 213, but the order of thehorizontal smoothing process and the vertical smoothing process may bereversed; the vertical smoothing processor 214 and second mixer 215 mayprecede the horizontal smoothing processor 212 and first mixer 213.Similar frame differences are still obtained.

To generalize, it is sufficient if the frame difference calculator (21)includes a subtractor (211) that obtains the signal difference betweenthe pixel of interest in the current frame and another pixel in the sameposition in the frame one frame before, a first horizontal smoothingprocessor (212 or 214) that performs smoothing in a first direction,which may be either the horizontal direction and the vertical direction,a first mixer (213 or 215) that mixes the signal smoothed by the firsthorizontal smoothing processor (212 or 214) with the output of thesubtractor (211), a second vertical smoothing processor (214 or 212)that performs smoothing in a second direction (the other direction) onthe output of the first mixer (213 or 215), and a second mixer (215 or213) that mixes the signal smoothed by the second vertical smoothingprocessor (214 or 212) with the output of the first mixer (213 or 215),and outputs a frame difference frd based on the output of the secondmixer (215 or 213).

The frame difference calculator 21 b in FIG. 13 may be used instead ofthe frame difference calculator 21 in FIG. 12. The frame differencecalculator 21 b in FIG. 13 performs the smoothing process similar tothat described in connection with FIG. 12 on the current frame signalDi0 and the one-frame delayed signal d2 f separately, and then obtainsthe frame difference value by performing a subtraction process on thesmoothed one-frame delayed signal d2 f and the smoothed current framesignal Di0.

In FIG. 13, components identical to or corresponding to components inFIG. 12 are denoted by the same reference numeral with a ‘b’ or ‘c’suffix. The horizontal smoothing processors 212 b, 212 c, mixers 213 b,213 c, vertical smoothing processors 214 b, 214 c, and mixers 215 b, 215c are similar in configuration to the horizontal smoothing processor212, mixer 213, vertical smoothing processor 214, and mixer 215 in FIG.12, but they receive video signal pixel values instead of differencevalues. The subtractor 211 b performs a subtraction process on the pixelvalue diff output from mixer 215 b and the pixel value dif2 output frommixer 215 c.

As in FIG. 12, the extent of the smoothing process is changed inaccordance with edge direction and pattern motion in the image and anabsolute value is taken to obtain a frame difference frd smoothed withinan appropriate extent.

The nonlinear conversion unit 22 in the frame difference detector 20 inFIG. 1 performs a nonlinear conversion on the frame difference frd fromthe frame difference calculator 21 (or 21 b) and generates a framedifference signal fdiff.

The nonlinear conversion is performed by multiplying the framedifference frd by a given sensitivity magnification factor Tmf,subtracting a given offset value Tof, and limiting the result to a givenrange (e.g., 0 to dM). The result is output to the motion informationcorrector 50 as the frame difference signal fdiff.

The frame difference detector 20 may output the frame difference frddirectly as the frame difference signal fdiff, in which case thenonlinear conversion unit 22 may be omitted.

FIG. 14 shows the input-output characteristic of the nonlinearconversion unit 22, showing an exemplary relationship between the inputframe difference frd (shown on the horizontal axis) and the output framedifference signal fdiff (shown on the vertical axis). In the exampleshown, when the frame difference frd is equal to or greater than a givenvalue Tm, the output frame difference signal fdiff has a constant valuedM. When the frame difference frd is equal to or less than an offsetvalue Tof, the frame difference signal fdiff is ‘0’, and is treated as aminor noise component.

When the frame difference frd is between the offset value Tof and thegiven value Tm, the frame difference signal fdiff increases from ‘0’ todM as the frame difference frd increases, indicating increasing amountsof motion or noise.

When the sensitivity magnification factor Tmf is increased, the framedifference frd is converted to a frame difference signal fdiff withlarger values, indicating greater amounts of motion or noise, so thatthe signal is more easily recognized as including motion or noiseexceeding a given level. Increasing the offset value Tof increases thevalue up to which the difference is treated as a minor noise component(a still part of the image). The sensitivity in detection of motion bymeans of the frame difference can accordingly be adjusted by adjustingthe sensitivity magnification factor Tmf and the offset value Tof.

If the sensitivity magnification factor Tmf is made too large, however,then even differences due to noise or unsteadiness in the video signal,which are smaller than differences due to motion, will be converted tomotion-like differences. The converted frame difference signal fdiffwill have such large values that even still parts of the image will bemistakenly detected as moving.

If the offset value Tof is too large, the converted frame differencesignal fdiff will be ‘0’ in parts that are actually moving, and thesemoving parts will be mistakenly detected as still (the motion will bemissed). The sensitivity magnification factor Tmf and the offset valueTof should be specified carefully to avoid these types of falsedetection.

Since the frame difference frd output from the frame differencecalculator 21 (or 21 b) is obtained by smoothing within an appropriateextent switched in accordance with edge direction and pattern motion inthe image and this switching can prevent the false detection of stillparts as moving due to noise etc. and the failure to detect motion wherepresent, the sensitivity magnification factor Tmf in the framedifference detector 20 need not be extremely large and the offset valueTof need not be extremely large or small.

Since the frame difference detector 20 obtains the frame differencehaving been smoothed within an appropriate extent responsive to edgedirection and pattern motion in the image by selecting the smoothingextent in the calculation of the frame difference fdiff in accordancewith the moving block coefficient dfmat and the edge detectioncoefficients efkm, the frame difference detector 20 can output anaccurate frame difference signal fdiff.

The motion information corrector 50 in FIG. 1 receives the framedifference signal fdiff from the frame difference detector 20, themoving block coefficients cbmat from the pattern matching detector 30,and the edge decision coefficients cedkm from the edge detector 40.

The motion information corrector 50 modifies the frame difference signalfdiff in accordance with the moving block coefficients cbmat and theedge decision coefficients cedkm and outputs the motion informationsignal md0 to the spatial and temporal expansion filtering unit 16.Modifying the frame difference signal fdiff in accordance with themoving block coefficients cbmat and the edge decision coefficients cedkmenables the frame difference signal fdiff to be corrected to a valuethat can be more recognized easily as moving in edges or patterns wheremoving areas tend to be misrecognized as still.

The motion information corrector 50 is configured as shown, for example,in FIG. 15.

The motion information corrector 50 in FIG. 15 includes a horizontalline motion adjuster 51, which performs a correction for horizontallines (vertical edges) moving vertically, and a vertical correlationadjuster 52, which performs a correction when vertically aligned pixelsas in a vertical line (horizontal edge) are strongly correlated. Thehorizontal line motion adjuster 51 includes a horizontal line motionadder 53 and a booster value calculator 54. The vertical correlationadjuster 52 includes a vertical correlation converter 55 and aconversion value calculator 56.

The second moving block coefficients cbmat input from the patternmatching detector 30 to the motion information corrector 50 include ahorizontal line moving block coefficient dfmat_mk reflecting verticalpattern motion and a vertical line moving block coefficient Vrdfmatreflecting horizontal pattern motion. The second edge decisioncoefficients cedkm input from the edge detector 40 include the verticaledge decision coefficient vedg_mk indicating the degree of presence of avertical edge component (horizontal line), the non-vertical edgedecision coefficient vrd_hvkm indicating the degree of absence of avertical edge, and the horizontal edge decision coefficient hedg_mkindicating the degree of presence of a horizontal edge component(vertical line).

The horizontal line motion adjuster 51 in FIG. 15 receives the framedifference signal fdiff from the frame difference detector 20, thevertical edge decision coefficient vedg_mk from the edge detector 40,and the horizontal line moving block coefficient dfmat_mk from thepattern matching detector 30.

The horizontal line motion adjuster 51 corrects the value of the framedifference signal fdiff in accordance with the vertical edge decisioncoefficient vedg_mk and the horizontal line moving block coefficientdfmat_mk, obtains corrected motion information mdad1 reflecting themotion of a horizontal line in the vertical direction, and outputs thisinformation to the vertical correlation adjuster 52 in the motioninformation corrector 50.

When an object with a repetitive pattern of horizontal lines moves inthe vertical direction, the smoothed differences are likely to be smalldespite the motion. Even though the frame difference detector 20 obtainsthe frame difference frd by switching the smoothing extent in accordancewith edge direction and pattern motion, if the difference value itselfis close to ‘0’, detection of the motion may fail.

The vertical edge decision coefficient vedg_mk from the edge detector 40indicates the degree of presence of a vertical edge component(horizontal line), and the horizontal line moving block coefficientdfmat_mk from the pattern matching detector 30 indicates the degree ofblock motion reflecting the vertical pattern motion. These twocoefficients enable the horizontal line motion adjuster 51 to detectvertical motion of an object with horizontal lines and to make acorresponding adjustment. For example, motion information based on theframe difference signal fdiff can be corrected by calculating a certainbooster value (offset value or first conversion value) had1 from thevertical edge decision coefficient vedg_mk and horizontal line movingblock coefficient dfmat_mk and adding this booster value had1 to theframe difference signal fdiff.

More specifically, the booster value calculator 54 in the horizontalline motion adjuster 51 calculates the booster value had1 for correctingthe frame difference signal fdiff in accordance with the second verticaledge decision coefficient vedg_mk in the second edge decisioncoefficients cedkm and the horizontal line moving block coefficientdfmat_mk in the second moving block coefficients cbmat.

For example, a certain value Adm1 may be set in the booster valuecalculator 54, and the booster value calculator 54 may multiply thevertical edge decision coefficient vedg_mk by the horizontal line movingblock coefficient dfmat_mk to obtain a coefficient k1 taking values from‘0’ to ‘1’ (by

k1=vedg _(—) mk×dfmat _(—) mk

for example), then multiply the set value Adm1 by coefficient k1 as inthe following equation (28) to obtain the booster value had1.

had1=k1×Adm1  (28)

When the horizontal line moving block coefficient dfmat_mk is ‘1’,indicating vertical pattern motion, and the vertical edge decisioncoefficient vedg_mk is ‘1’, indicating the presence of a vertical edgecomponent, coefficient k1 is ‘1’, indicating a moving pattern that has ahorizontal line component or vertical edge and is moving in the verticaldirection.

If the value Adm1 is converted in accordance with coefficient k1 as inequation (28) above, when coefficient k1 is ‘1’, the value Adm1 itselfis output as the booster value hadl, and when coefficient k1 is ‘0’, thebooster value had1 is ‘0’. When coefficient k1 is between ‘0’ and ‘1’,the value Adm1 is converted in accordance with coefficient k1 and outputas the booster value had1. When an object with horizontal lines movesvertically, the booster value had1 is generated with its maximum valueof Adm1 in accordance with the vertical edge decision coefficientvedg_mk and horizontal line moving block coefficient dfmat_mk.

The booster value had1 need not be obtained by the multiplication by thecoefficient k1 described above. It may be obtained by selecting the setvalue Adm1 when both the horizontal line moving block coefficientdfmat_mk and the vertical edge decision coefficient vedg_fk are equal toor greater than predetermined values (such as ‘1’) and selecting ‘0’otherwise. Alternatively, values of the booster value had1 correspondingto values of the horizontal line moving block coefficient dfmat_mk andvertical edge decision coefficient vedg_fk may be specified in a LUT,which is then used to convert the horizontal line moving blockcoefficient dfmat_mk output from the pattern matching detector 30 andthe vertical edge decision coefficient vedg_fk output from the edgedetector 40 to a corresponding value had1. The multiplication operationdescribed above, however, enables the booster value to be switchedsmoothly, so that a more appropriate booster value had1 can be obtained.

The horizontal line motion adder 53 receives the frame difference signalfdiff from the frame difference detector 20 and the booster value had1from the booster value calculator 54. The horizontal line motion adder53 corrects the frame difference signal fdiff by adding the boostervalue had1 to the frame difference signal fdiff, thereby generatingcorrected motion information mdad1 that has been boosted in thedirection of motion detection (in such a direction that motion is moreeasily recognized) when a horizontal line moves in the verticaldirection.

Instead of having the horizontal line motion adder 53 in the horizontalline motion adjuster 51 make the correction by adding the booster valuehad1 to the frame difference signal fdiff, the frame difference signalfdiff may be multiplied by a booster coefficient (greater than 1) thatvaries according to the moving block coefficient dfmat_mk and verticaledge decision coefficient vedg_fk. Alternatively, a predetermined valueindicative of motion (for example, the maximum value indicating motion)may be selectively output instead of the frame difference signal fdiff.A similar effect can be obtained by any correction that adjusts thevalue of the frame difference signal fdiff in the direction of motiondetection (in such a direction that motion is more easily recognized) inaccordance with the values of the moving block coefficient dfmat_mk andvertical edge decision coefficient vedg_fk, (adjusting the value to avalue that is more easily detected as motion) and thereby obtainscorrected motion information mdad1 that takes account of vertical motionof horizontal lines.

The horizontal line motion adjuster 51 can thus provide corrected motioninformation mdad1 that has been altered in accordance with the verticaledge decision coefficient vedg_mk and horizontal line moving blockcoefficient dfmat_mk to facilitate the detection of motion when ahorizontal line moves in the vertical direction, and can prevent thefalse detection of nonexistent motion and the failure to detect realmotion.

The vertical correlation adjuster 52 in the motion information corrector50 receives the corrected motion information mdad1 from the horizontalline motion adjuster 51, the horizontal edge decision coefficienthedg_mk and the non-vertical edge decision coefficient vrd_hvkm from theedge detector 40, and the vertical line moving block coefficient Vrdfmatfrom the pattern matching detector 30.

The vertical correlation adjuster 52 modifies the value of the correctedmotion information mdad1 based on the frame difference signal fdiff inaccordance with the horizontal edge decision coefficient hedg_mk,non-vertical edge decision coefficient vrd_hvkm, and vertical linemoving block coefficient Vrdfmat and generates corrected motioninformation mdv2 that takes account of the motion of verticallycorrelated pixels, as in a moving vertical line. The corrected motioninformation mdv2 is output as the motion information signal md0 from themotion information corrector 50 to the spatial and temporal expansionfiltering unit 16.

When an object with a repetitive pattern of vertical lines moves in thehorizontal direction, the smoothed differences are likely to beparticularly small despite the motion. Even though the frame differencedetector 20 obtains the frame difference frd by switching the smoothingextent in accordance with edge direction and pattern motion, if thedifference value itself is close to ‘0’, detection of the motion mayfail.

The horizontal edge detection coefficient hedg_mk from the edge detector40 indicates the degree of presence of a horizontal edge component(vertical line), the non-vertical edge detection coefficient vrd_hvkmindicates the degree of absence of a vertical edge (the degree ofpresence of a pattern that is not a vertical edge or horizontal line),and the vertical line moving block coefficient Vrdfmat from the patternmatching detector 30 indicates the degree of block motion reflecting thehorizontal pattern motion.

These three coefficients enable the vertical correlation adjuster 52 todetect horizontal motion of an object with vertical lines and make acorresponding adjustment. For example, the corrected motion informationmdad1 may be further corrected by calculating a certain booster value(offset value) ad2 and magnification factor m12 (greater than ‘1’) fromthe horizontal edge detection coefficient hedg_mk, non-vertical edgedecision coefficient vrd_hvkm, and vertical line moving blockcoefficient Vrdfmat, then adding the booster value ad2 and multiplyingthe sum by the magnification factor m12.

More specifically, the conversion value calculator 56 in the verticalcorrelation adjuster 52 calculates conversion coefficients (secondconversion values) vdmv including the booster value (offset value) ad2and magnification factor m12 for correcting the corrected framedifference signal or motion information mdad1 in accordance with thehorizontal edge decision coefficient hedg_mk and non-vertical edgedecision coefficient vrd_hvkm in the second edge decision coefficientscedkm and the vertical line moving block coefficient Vrdfmat in thesecond moving block coefficients cbmat.

For example, a certain value Adm2 and a magnification factor Md2 may beset in the conversion value calculator 56 and the conversion valuecalculator 56 may multiply the horizontal edge decision coefficienthedg_mk, non-vertical edge decision coefficient vrd_hvkm, and verticalline moving block coefficient Vrdfmat together to obtain a coefficientk2 taking values from ‘0’ to ‘1’ (by the operation

k2=hedg _(—) mk×Vrdfmat×vrd _(—) hvkm

for example), then multiply the set value Adm2 and magnification factorMd2 by coefficient k2 as in equations (29) and (30) below to obtain thebooster value ad2 and the magnification factor m12 that constitute theconversion coefficients vdmv.

ad2=k2×Adm2  (29)

m12=k2×Md2  (30)

When the vertical line moving block coefficient Vrdfmat is ‘1’,indicating horizontal pattern motion, the horizontal edge decisioncoefficient hedg_mk is ‘1’, indicating the presence of a horizontal edgecomponent, and the non-vertical edge decision coefficient vrd_hvkm is‘1’, indicating the absence of a vertical edge component, coefficient k2is ‘1’, indicating horizontal pattern motion in the presence of ahorizontal edge or vertical line component alone. If the value Adm2 andmagnification factor Md2 are converted in accordance with coefficient k2as in equations (29) and (30), when coefficient k2 is ‘1’, the valueAdm2 itself is output as the booster value ad2, and magnification factorMd2 itself is output as magnification factor mdl2. When coefficient k2is ‘0’, the booster value ad2 and magnification factor m12 are both ‘0’.

When coefficient k2 is between ‘0’ to ‘1’, the Adm2 value andmagnification factor Md2 are converted in accordance with coefficient k2and output as the booster value ad2 and magnification factor m12.Accordingly, when an object with only vertical lines moves in thehorizontal direction, the booster value ad2 and magnification factor m12are generated with respective maximum values of Adm2 and Md2 inaccordance with the horizontal edge decision coefficient hedg_mk,non-vertical edge decision coefficient vrd_hvkm, and vertical linemoving block coefficient Vrdfmat. The booster value ad2 andmagnification factor m12 thus generated are output from the conversionvalue calculator 56 as the conversion coefficients vdmv and supplied tothe vertical correlation converter 55.

These conversion coefficients vdmv need not be obtained bymultiplication by the coefficient k2 described above. They may beobtained by selecting the set value Adm2 and magnification factor Md2when the horizontal edge decision coefficient hedg_mk, the non-verticaledge decision coefficient vrd_hvkm, and the vertical line moving blockcoefficient Vrdfmat are equal to or greater than a given value (such as‘1’) and selecting ‘0’ otherwise.

Alternatively, values of the conversion coefficients vdmv correspondingto values of the horizontal edge decision coefficient hedg_mk,non-vertical edge decision coefficient vrd_hvkm, and vertical linemoving block coefficient Vrdfmat may be specified in a LUT, which isthen used to convert the input horizontal edge decision coefficienthedg_mk, non-vertical edge decision coefficient vrd_hvkm, and verticalline moving block coefficient Vrdfmat to the corresponding vdmv values.The multiplication operations described above, however, enable theconversion coefficients vdmv to be switched smoothly, so that moreappropriate conversion coefficients vdmv can be obtained.

The coefficient k2 need not be obtained by using the horizontal edgedecision coefficient hedg_mk, non-vertical edge decision coefficientvrd_hvkm, and vertical line moving block coefficient Vrdfmat. Acoefficient k2 indicating motion of an object with vertical lines in thehorizontal direction may be obtained by excluding the non-vertical edgedecision coefficient vrd_hvkm. In that case, even if both vertical andhorizontal line components are detected, when the vertical line movingblock coefficient Vrdfmat indicates a moving pattern, the conversioncoefficients vdmv will have non-zero values.

The vertical correlation converter 55 receives the corrected motioninformation mdad1 from the horizontal line motion adjuster 51 and theconversion coefficients vdmv from the conversion value calculator 56.The vertical correlation converter 55 corrects the value of thecorrected motion information mdad1 by a scaling operation thatmultiplies the corrected motion information mdad1 by the magnificationfactor m12 included in the conversion coefficients vdmv, and then byadding the booster value ad2 to generate corrected motion informationmdv2 that has been altered in the direction of motion detection (in sucha direction that motion is more easily recognized) when pixels having a(strong) vertical correlation as in a vertical line move in thehorizontal direction. The corrected motion information mdv2 is outputfrom the motion information corrector 50 as the motion informationsignal md0.

Instead of making the correction by a conversion involving both abooster value ad2 and a magnification factor m12, the conversion valuecalculator 56 in the vertical correlation adjuster 52 may use only oneof these two, either the booster value ad2 or the magnification factorm12, to effect a conversion in the direction of enlarging the framedifference signal.

Alternatively, a predetermined value indicative of motion (for example,the maximum value indicating motion) may be selectively output insteadof the corrected motion information mdad1. A similar effect can beobtained by any correction that adjusts the value of the correctedmotion information mdad1 indicating the frame difference in thedirection of motion detection (in such a direction that motion is moreeasily recognized) in accordance with the values of the horizontal edgedecision coefficient hedg_mk, non-vertical edge decision coefficientvrd_hvmk, and vertical line moving block coefficient Vrdfmat (adjustingthe value to a value that is more easily detected as motion) and therebyobtains corrected motion information mdv2 that takes account ofhorizontal motion of vertically correlated pixels, such as the pixels ina vertical line.

The vertical correlation adjuster 52 can thus provide corrected motioninformation mdv2 that has been altered in accordance with the horizontaledge decision coefficient hedg_mk, non-vertical edge decisioncoefficient vrd_hvkm, and vertical line moving block coefficient Vrdfmatto facilitate the detection of motion when an object with verticalcorrelation moves horizontally, and can prevent false detection ofnonexistent motion and failure to detect real motion.

When a horizontal line moves in the vertical direction or when a movingpattern possesses vertical correlation, the motion information corrector50 can thus correct the frame difference signal fdiff in accordance withthe moving block coefficients cbmat from the pattern matching detector30 and the edge decision coefficients cedkm from the edge detector 40 tofacilitate the detection of motion, thereby obtaining a motioninformation signal md0 that takes account of edge direction and patternmotion. A highly accurate motion information signal md0 can therefore beobtained, preventing the false detection of nonexistent motion andfailure to detect real motion.

Although the correction by the vertical correlation adjuster 52 iseffected after the correction by the horizontal line motion adjuster 51in the motion information corrector 50 in FIG. 15, the correction by thevertical correlation adjuster 52 may precede the correction by thehorizontal line motion adjuster 51, or just one of the two correctionsmay be made.

Referring again to FIG. 1, the spatial and temporal expansion filteringunit 16 receives the motion information signal md0 from the motioninformation corrector 50. The spatial and temporal expansion filteringunit 16 performs processes such as spatial and temporal filtering andisolated point removal on the motion information signal md0, obtains amotion signal afmd by correcting the motion information in the spatialand temporal directions, and outputs it to the motion detection signalconverter 19.

The temporal filtering section 17 in the spatial and temporal expansionfiltering unit 16 performs a temporal expansion process on the inputmotion information signal md0 and make a correction that prevents falsedetection of nonexistent motion and failure to detect real motion.

The spatial filtering section 18 in the spatial and temporal expansionfiltering unit 16 performs spatial filtering using a horizontal orvertical LPF, isolated point removal, and other filtering processes onthe motion information signal to correct mistakenly detected nonexistentmotion and real but non-detected motion in terms of the motion of pixelson the screen, and outputs a motion signal afmd.

The motion signal afmd processed by the spatial and temporal expansionfiltering unit 16 has a value that is large when motion is large and theframe difference is detected as motion, and decreases to a small valueclose to ‘0’ for still parts or noise. This value indicates the amountof image motion at the pixel undergoing motion detection.

The motion detection signal converter 19 receives the motion signal afmdfrom the spatial and temporal expansion filtering unit 16. The motiondetection signal converter 19 converts the motion signal afmd to amotion detection signal mds indicating the degree of motion in the videosignal.

FIG. 16 shows an exemplary input-output characteristic of the motiondetection signal converter 19, illustrating the conversion of the motioninformation afmd (horizontal axis, input) to the motion detection signalmds (vertical axis, output), in which the motion detection signal mds isconverted to a value ranging from ‘0’ to ‘16’, indicating differentdegrees of motion in the video signal. The value of the motion detectionsignal mds is obtained by a nonlinear conversion in accordance with thevalue of the motion signal afmd. A value of the motion signal afmd thatis large and indicates motion (the highest degree of motion) isconverted to ‘16’, the maximum value of the motion detection signal mds.A signal indicating a still image is converted to a ‘0’ value of themotion detection signal mds. The conversion may be carried out by use ofa LUT provided to convert the motion signal afmd to the correspondingvalue of the motion detection signal mds, or by a nonlinear conversioninvolving operations such as multiplication of the motion signal afmd bygiven magnification factors and/or adding or subtracting values. Theexemplary motion detection signal mds takes values ranging from ‘0’ to‘16’, but it may have lower resolution, taking values ranging from ‘0’to ‘8’, for example, or higher resolution. Alternatively, the value ofthe motion signal afmd may be output directly as the value of the motiondetection signal mds, without performing a nonlinear conversion. Thesignal may be a binary signal indicating motion as ‘1’ and still partsas ‘0’.

In the motion detection device 1 shown in FIG. 1, the pattern matchingdetector 30 generates moving block coefficients dfmat, dbmat by patternmatching to indicate pattern motion in the pixel blocks. The edgedetector 40 generates edge decision coefficients efkm, cedkm indicatingthe degree of presence of edge components. The frame difference detector20 generates a frame difference by selecting an appropriate smoothingextent in accordance with the generated block coefficients dfmat andedge decision coefficients efkm, which indicate pattern motion and edgedirection in the image, and performing smoothing within the selectedextent. The motion information corrector 50 generates a motioninformation signal md0 corrected in accordance with edge direction andpattern motion according to the generated moving block coefficientscbmat and edge decision coefficients cedkm.

Therefore, the motion detection device 1 can provide a motion detectionsignal mds indicating amounts of motion with high accuracy, withoutmistakenly detecting still parts as moving and without failing to detectmoving parts.

A flowchart description of the operation of the video signal processingdevice in the first embodiment will now be given. Briefly, the framedelay unit 13 obtains a frame delayed signal and the motion detectiondevice 1 obtains a frame difference signal indicating a smootheddifference between the delayed and undelayed video signals. The extentof smoothing in the frame differencing process is switched in accordancewith edge direction and pattern motion, according to moving blockcoefficients obtained by pattern matching and edge decision coefficientsobtained by edge detection. The frame difference signal is corrected inaccordance with edge direction and pattern motion, and a motiondetection signal mds indicating degrees of motion is output.

FIGS. 17 and 18 constitute a flowchart illustrating the operation of themotion detection device 1 in detecting image motion pixel by pixel inthe video signal processing device in the first embodiment.

The frame delay unit 13 delays the input current frame signal Di0 andobtains a one-frame delayed signal d2 f and one-field delayed signal d1f. The motion detection device 1 receives the current frame signal Di0,one-frame delayed signal d2 f, and one-field delayed signal d1 f (stepS101).

The pattern matching detector 30 receives the current frame signal Di0and the one-frame delayed signal d2 f, calculates pattern similaritybetween the blocks in the one-frame delayed signal d2 f and the currentframe signal Di0, and generates moving block coefficients dfmat, cbmatindicating motion of the pixel blocks. The moving block coefficientsdfmat, cbmat are obtained as results of pattern matching, and indicatepixel block motion and pattern motion (step S102).

The edge detector 40 detects whether there is an edge, or a part ofcontour of an object, in the vicinity of the pixel of interest P0, fromthe current frame signal Di0, one-frame delayed signal d2 f, andone-field delayed signal d1 f, and generates edge detection coefficientsefkm, cedkm (step S103).

In the frame difference detector 20, the subtractor 211 in the framedifference calculator 21 calculates a frame-to-frame difference Dnbetween the current frame signal Di0 and the one-frame delayed signal d2f (step S104); the horizontal smoothing processor 212 smoothes thedifference Dn to obtain a horizontally smoothed difference hlpd; themixer 213 mixes the difference Dn and the horizontally smootheddifference hlpd in accordance with the horizontal edge decisioncoefficient hedg_fk in the edge decision coefficients efkm and obtains ahorizontally mixed difference hdf.

The horizontally mixed difference hdf is generated by, for example,using the horizontal edge decision coefficient hedgfk as a mixing ratioto mix the difference Dn and the horizontally smoothed difference hlpd(step S105). This mixing operation is expressed by the followingequation.

hdf=(1−hedg _(—) fk)×hlpd+hedg _(—) fk×Dn

The vertical smoothing processor 214 obtains a vertically smootheddifference vlpd by smoothing the horizontally mixed difference hdf; themixer 215 mixes the horizontally mixed difference hdf and the verticallysmoothed difference vlpd in accordance with the moving block coefficientdfmat and the vertical edge decision coefficient vedg_fk in the edgedecision coefficients efkm to generate a mixed difference dif. Forexample, a conversion to a coefficient vdk ranging from ‘0’ to ‘1’ iseffected by multiplying the vertical edge decision coefficient vedg_fkby the moving block coefficient dfmat (step S106). The value of thecoefficient vdk increases as the degree of presence of a vertical edgecomponent (horizontal line) increases and as the degree of motionincreases.

The horizontally mixed difference hdf and the vertically smootheddifference vlpd are mixed, using the coefficient vdk as a mixing ratio,to generate the mixed difference dif (step S107). The mixing process isexpressed by the following equation.

dif=(1−vdk)×vlpd+vdk×hdf

A frame difference signal fdiff is obtained through calculation of theabsolute value of the mixed difference dif in the absolute valuecalculator 216 and a nonlinear conversion performed by the nonlinearconversion unit 22 (step S108).

Since the smoothing extent is selected in accordance with the movingblock coefficient dfmat and the edge decision coefficients efkm in stepsS102, S103, and S104 to S108, the frame difference is smoothed within anappropriate extent depending on edge direction and pattern motion, andthe frame difference detector 20 outputs a highly accurate framedifference signal fdiff.

The frame difference signal fdiff output from the frame differencedetector 20 is corrected by the motion information corrector 50 andoutput as a motion information signal md0. The frame difference signalfdiff is modified in accordance with the moving block coefficients cbmatand the edge decision coefficients cedkm to obtain a value thatfacilitates the detection of motion at edges or patterns that tend to bedetected mistakenly as being still (steps S109, S110).

More specifically, the horizontal line motion adjuster 51 in the motioninformation corrector 50 calculates a given booster value from thevertical edge decision coefficient vedg_mk and the horizontal linemoving block coefficient dfmat_mk and adds the booster value to theframe difference signal fdiff, thereby correcting the motion informationbased on the frame difference signal when there is vertical motion of ahorizontal line, for example, and generates corrected motion informationmdad1 (step S109).

The vertical correlation adjuster 52 in the motion information corrector50 calculates a given booster value and a magnification factor (greaterthan ‘1’) from a horizontal edge decision coefficient hedg_mk,non-vertical edge decision coefficient vrd_hvkm, and vertical linemoving block coefficient Vrdfmat, converts the corrected motioninformation mdad1 by adding the booster value and multiplying by themagnification factor, and generates a motion information signal md0(step S110). The corrected value facilitates the detection of movingpatterns with vertical but not horizontal lines.

The motion information corrector 50 corrects the frame difference signalfdiff in accordance with the moving block coefficients cbmat and edgedecision coefficients cedkm to a value that facilitates detection ofvertical motion of horizontal lines and the detection of moving patternswith vertical correlation, obtaining a motion information signal md0corrected in accordance with edge direction and pattern motion.

The spatial and temporal expansion filtering unit 16 performs a temporalexpansion process on the motion information signal md0 from the motioninformation corrector 50 (step S111) and performs spatial filtering byusing a horizontal or vertical LPF, by isolated point removal, or byother filtering processes (step S112).

The motion detection signal converter 19 converts the motion informationafmd from the spatial and temporal expansion filtering unit 16 to amotion detection signal mds indicating the degree of motion in the videosignal (step S113).

The motion detection device 1 outputs the motion detection signal mds asthe result of motion detection (step S114). The motion detection device1 thus generates, by pattern matching, moving block coefficients dfmat,cbmat indicating pattern motion in the pixel blocks, and generates edgedecision coefficients efkm, cedkm indicating the degree of presence ofedge components. By performing a smoothing process within an extentselected in accordance with the moving block coefficient dfmat and theedge decision coefficients efkm, it obtains a frame difference smoothedwithin a suitable extent responsive to edge direction and patternmotion. Furthermore, a motion information signal md0 corrected accordingto edge direction and pattern motion is obtained in accordance with themoving block coefficients cbmat and the edge decision coefficientscedkm.

Therefore, the motion detection device 1 can obtain a motion detectionsignal mds indicating amounts or degrees of motion accurately even froman image with noise, without failing to detect motion and withoutmistakenly detecting still parts as moving.

In the video signal processing device in the first embodiment, thepattern matching detector 30 uses pattern matching to generate movingblock coefficients dfmat, cbmat that indicate the pattern motion inpixel blocks; the edge detector 40 generates edge decision coefficientsefkm, cedkm that indicate the degree of presence of an edge component;the frame difference detector 20 obtains a frame difference smoothedwithin a suitable extent, responsive to edge direction and patternmotion, by performing a smoothing process within an extent selected inaccordance with the moving block coefficient dfmat and edge decisioncoefficients efkm. The motion information corrector 50 obtains a motioninformation signal md0 corrected in accordance with edge direction andpattern motion according to the moving block coefficients cbmat and edgedecision coefficients cedkm.

Therefore, the smoothing extent used in obtaining the frame differencecan be changed in accordance with edge direction and pattern motion, byusing moving block coefficients indicating block motion as determinedfrom the value of pattern similarity between pixel blocks and edgedecision coefficients indicating the degree of edge presence, and aframe difference signal smoothed in an appropriate extent can beobtained. The frame difference signal is corrected in accordance withedge direction and pattern motion by using the moving block coefficientsand the edge decision coefficients, and at an edge or pattern wheremotion is likely to be detected mistakenly as being still, a correctioncan be made to facilitate motion detection, so that motion in an imagecontaining noise can be detected accurately without mistakenly detectingstill parts as moving and without failing to detect motion in movingparts.

In the video signal processing device in the first embodiment, the edgedetector 40 shown in FIG. 10 detects edges by extracting vertical edgecomponents and horizontal edge components in the vicinity of the pixelof interest P0 from the current frame signal Di0, one-frame delayedsignal d2 f, and one-field delayed signal d1 f. This configuration canbe changed to detect edges by extracting edge components from thecurrent frame signal Di0 and the one-frame delayed signal d2 f, whichare in phase, without using the one-field delayed signal d1 f. In thatcase, the vertical high frequency component extractor 413 and horizontalhigh frequency component extractor 423 in FIG. 10 may be eliminated, oralternatively, the vertical high frequency component selector 414 andhorizontal high frequency component selector 424 may selectively blockthe signals from the vertical high frequency component extractor 413 andhorizontal high frequency component extractor 423.

The video signal processing device in the first embodiment receives aninterlaced video signal and detects motion in the video signal betweenthe current frame Di0 and the immediately preceding frame d2 f. Even ifthe input video signal is a progressive signal, however, motion can bedetected with the same configuration. When a progressive signal isinput, the frame delay unit 13 delays the input current frame signal Di0by one frame and outputs the signal d2 f of the preceding frame. Sincethe out-of-phase signal dlf of the preceding field is not obtained, theedge detector 40 detects edges from the in-phase signal d2 f of thepreceding frame.

Although a pixel P0 in the current frame signal Di0 is the pixel ofinterest for which motion is detected in the video signal processingdevice in the first embodiment, the pixel of interest P0 need not bedisposed in the current frame; motion detection may be centered on aframe of a signal that has been delayed by a field or a frame, and thepixel of interest P0 may be disposed in that delayed signal. In thatcase, the one-frame difference detected by the motion detection device 1may be a difference involving the frame preceding the central frame onthe time axis or the two frames following the central frame on the timeaxis. It is also possible to combine differences with frames disposed inboth the preceding and following directions. Provided the motiondetection signal is obtained by detecting differences between theseframes and the central frame, the same effects as described above in thefirst embodiment are obtained. More generally, the motion detectiondevice of the present invention detects motion from temporally differingfirst and second frames of the video signal, with a pixel in the firstframe as the pixel of interest. The first frame may be the current frame(the latest frame) or a frame preceding the current frame by one frameor one field.

The constituent units of the frame delay unit and motion detectiondevice 1 may be configured either as hardware or as software, in theform of a programmed computing device.

Second Embodiment

Whereas the video signal processing device in the first embodiment wasconfigured to detect pixel motion in the image from the current framesignal and the signal one frame before and obtained frame differencesbetween the current frame signal and the signal one frame before in itsframe difference detector 20, the video signal processing device shownin FIG. 19 is configured to detect pixel motion in the image byobtaining a frame difference signal from frame differences between thecurrent frame signal and the signal two frames before (two-framedifferences) as well as from frame differences between the current framesignal and the signal one frame before.

FIG. 19 is a block diagram showing the configuration of a video signalprocessing device of the second embodiment of the invention (device forimplementing the video signal processing method of the second embodimentof the invention). The device is configured to detect image motion pixelby pixel from the current frame signal Di0, the signal d2 f one framebefore, and the signal d4 f two frames before. Elements in FIG. 19 thatare identical to or correspond to elements shown in FIG. 1 in the firstembodiment have the same reference characters.

The video signal processing device in the second embodiment in FIG. 19includes a pair of frame delay units 13, 15 that delay the video signalinput as the current frame signal Di0 by one frame apiece, and a motiondetection device 2 that detects pixel motion from the current framesignal Di0 and the resulting delayed signals d2 f, d4 f.

The motion detection device 2 is generally similar to the motiondetection device 1 shown in FIG. 1, but uses a different framedifference detector 20 b and a different pattern matching detector 31instead of the frame difference detector 20 and pattern matchingdetector 30 shown in FIG. 1.

The frame difference detector 20 b detects two-frame differences inaddition to one-frame differences, and outputs a frame difference signalfor detecting motion.

The pattern matching detector 31 generates moving block coefficientsfrom the similarity of patterns in pixel blocks in the signal of thepreceding frame and the signal of the frame preceding that frame (theframe two frames before), as well as from the similarity of patterns inpixel blocks in the signal of the current frame and the signal of thepreceding frame.

The spatial and temporal expansion filtering unit 16, motion detectionsignal converter 19, edge detector 40, and motion information corrector50 are identical in configuration to the corresponding elements in thefirst embodiment. Detailed descriptions will be omitted.

The frame difference calculators 21 c, 21 d that calculate framedifferences in the frame difference detector 20 b in FIG. 19 areidentical to the frame difference calculator 21 in FIG. 1, but whereasframe difference calculator 21 c receives the current frame signal Di0and one-frame delayed signal d2 f, frame difference calculator 21 dreceives the one-frame delayed signal d2 f and two-frame delayed signald4 f. The two nonlinear conversion units 22 c, 22 d are both identicalto the nonlinear conversion unit 22 in FIG. 1. The frame differencedetector 20 b in FIG. 10 also includes a difference combiner 23 forcombining the one-frame difference signal fdiff1 output from nonlinearconversion unit 22 c and the two-frame difference signal fdiff2 outputfrom nonlinear conversion unit 22 d.

FIG. 19 will now be used to describe motion detection in the secondembodiment.

The frame delay units 13, 15 both output a video signal that has beendelayed by one frame. Frame delay unit 13 delays the input video signalDi0 by one frame and outputs the one-frame delayed signal d2 f. Framedelay unit 15 delays the one-frame delayed signal d2 f by one frame andoutputs the two-frame delayed signal d4 f. When an interlaced signal isinput, the frame delay units 13, 15 may be configured from pairs offield memories that impart a one-field delay, and a one-field delayedsignal d1 f may be obtained from frame delay unit 13, as shown. Eachframe delay unit 13, 15 is identical in configuration and operation tothe frame delay unit 13 in FIG. 1.

The two-frame delayed signal d4 f output from frame delay unit 15 is,like the one-frame delayed signal d2 f, an in-phase signal having pixelsin the same positions as in the current frame signal Di0.

The current frame signal Di0 is input to the motion detection device 2together with the one-frame delayed signal d2 f and one-field delayedsignal d1 f from frame delay unit 13 and the two-frame delayed signal d4f from frame delay unit 15. The motion detection device 2 uses thecurrent frame signal Di0, one-frame delayed signal d2 f, one-fielddelayed signal d1 f, and two-frame delayed signal d4 f to detect pixelmotion and output a motion detection signal mds that indicates degreesof motion based on the detection results.

The motion detection device 2 calculates moving block coefficients bypattern matching of pixel blocks in the one-frame delayed signal d2 fand two-frame delayed signal d4 f, as well as by pattern matching ofpixel blocks in the current frame signal Di0 and one-frame delayedsignal d2 f, detects edge direction from the moving block coefficientsand from edge decision coefficients resulting from edge detection, andobtains a frame difference signal that has been smoothed over a suitableextent, the extent being switched according to edge direction andpattern motion. By detecting frame differences from both the one-framedifferences between the current frame signal Di0 and one-frame delayedsignal d2 f and the two-frame differences between the current framesignal Di0 and two-frame delayed signal d4 f, the motion detectiondevice 2 performs highly accurate motion detection with betterprevention of false detection of nonexistent motion and failure todetect real motion, and generates and outputs a motion detection signalmds indicating the degree of motion.

The configuration and operation of the spatial and temporal expansionfiltering unit 16, motion detection signal converter 19, edge detector40, and motion information corrector 50 in the motion detection device 2are as shown in FIG. 1 and described above, so that detaileddescriptions will be omitted. The following description concerns thegeneration of the dfmat and moving block coefficients cbmat by thepattern matching detector 31 and the generation of the frame differencesignal sfdiff in the frame difference detector 20 b.

The current frame signal Di0, one-frame delayed signal d2 f, andtwo-frame delayed signal d4 f are input to the pattern matching detector31 in the motion detection device 2.

The pattern matching detector 31 carries out pattern matching tocalculate the similarity of pixel blocks in the one-frame delayed signald2 f and the current frame signal Di0, and the similarity of pixelblocks in the one-frame delayed signal d2 f and the two-frame delayedsignal d4 f. From these two pattern matching results, the moving blockcoefficients dfmat, cbmat that indicate pixel block motion aregenerated. The detected moving block coefficients dfmat, cbmat areobtained from the pattern matching results for pixel blocks in the sameposition as the pixel of interest, the motion of which is beingdetected. Increasing similarity values indicate increasing dissimilaritybetween the patterns in the pixel blocks, thereby indicating pixel blockmotion, or motion of the block patterns in the pixel blocks (patternmotion).

The pattern matching detector 31 is configured as shown, for example, inFIG. 20A.

Elements in FIG. 20A that are identical to or correspond to elements ofthe pattern matching detector 30 shown in FIG. 2A in the firstembodiment have the same reference characters.

Of the constituent elements of the pattern matching detector 31 in FIG.20A, the block matching operation units 301 a and 301 b are identical tothe block matching operation unit 301 in pattern matching detector 30.The matching vertical high frequency component generators 308 a, 308 b,which include the horizontal block matching operation units 303 a, 303 band matching vertical high frequency component extractors 304 a, 304 b,as shown in FIGS. 20B and 20C, and the matching horizontal highfrequency component generators 309 a, 309 b, which include the verticalblock matching operation units 306 a, 306 b and matching horizontal highfrequency component extractors 307 a, 307 b, as shown in FIG. 20D andFIG. 20E, are identical to the matching vertical high frequencycomponent generator 308, which includes the horizontal block matchingoperation unit 303 and matching vertical high frequency componentextractor 304, and the matching horizontal high frequency componentgenerator 309, which includes the vertical block matching operation unit306 and matching horizontal high frequency component extractor 307, inpattern matching detector 30.

Pattern matching detector 31 also has a maximum value selector 321, avertical high frequency maximum value selector 322, and a horizontalhigh frequency maximum value selector 323.

Block matching operation unit 301 a in FIG. 20A receives the currentframe signal Di0 and the one-frame delayed signal d2 f, performs blockmatching on a pixel block b0 centered on the pixel of interest P0 in thecurrent frame signal Di0 and a corresponding pixel block b2 in the frameof the one-frame delayed signal d2 f, and obtains as a result a blockmatching quantity blkpm1 indicating the similarity of the patterns inthese blocks. Block matching operation unit 301 b receives the one-framedelayed signal d2 f and the two-frame delayed signal d4 f, performsblock matching on a pixel block b2 in the frame of the one-frame delayedsignal d2 f and a corresponding pixel block b4 in the frame of thetwo-frame delayed signal d4 f, and obtains as a result a block matchingquantity blkpm2 indicating the similarity of the patterns in theseblocks.

The block matching quantity blkpm1 calculated by block matchingoperation unit 301 a and the block matching quantity blkpm2 calculatedby block matching operation unit 301 b are sent to the maximum valueselector 321, which selects the larger of the two block matchingquantities blkpm1, blkpm2 and outputs it as a block matching quantityblkpm. This block matching quantity blkpm is output to the first,second, and third moving block coefficient conversion units 311, 312,313 as a result indicating the similarity of the pixel block b0 centeredon the pixel of interest P0 to blocks in two preceding frames, thusindicating motion of the block pattern in the pixel block.

Instead of performing pattern matching on the one-frame delayed signald2 f and two-frame delayed signal d4 f to obtain the block matchingquantity blkpm2, block matching operation unit 301 b may be configuredto perform pattern matching on the current frame signal Di0 andtwo-frame delayed signal d4 f, which still enables the maximum valueselector 321 to obtain as a result a block matching quantity blkpm overa two-frame interval.

Horizontal block matching operation unit 303 a receives the currentframe signal Di0 and the one-frame delayed signal d2 f and carries outpattern matching on horizontally elongated sub-block areas constitutingparts of the pixel blocks in these frames.

Horizontal block matching operation unit 303 b receives the one-framedelayed signal d2 f and the two-frame delayed signal d4 f and carriesout pattern matching on horizontally elongated sub-block areasconstituting parts of the pixel blocks in these frames.

From the horizontal block matching quantities hp11, hp12, hpl3 output byhorizontal block matching operation unit 303 a, matching vertical highfrequency component extractor 304 a obtains a matching vertical highfrequency component vebpm1. From the horizontal block matchingquantities hp21, hp22, hp23 output by horizontal block matchingoperation unit 303 b, matching vertical high frequency componentextractor 304 b obtains a matching vertical high frequency componentvebpm2.

The two matching vertical high frequency components vebpm1 and vebpm2calculated by the matching vertical high frequency component extractors304 a and 304 b are sent to the vertical high frequency maximum valueselector 322. The vertical high frequency maximum value selector 322selects the largest of the two matching vertical high frequencycomponents vebpm1 and vebpm2 and outputs it as the matching verticalhigh frequency component vebpm. This matching vertical high frequencycomponent vebpm is output to the first and second moving blockcoefficient conversion units 311 and 312 as a value indicating thepresence of a vertical motion component in the pixel blocks over atwo-frame interval.

Vertical block matching operation unit 306 a receives the current framesignal Di0 and the one-frame delayed signal d2 f and carries out patternmatching on vertically elongated sub-block areas constituting parts ofthe pixel blocks in the respective frames.

Vertical block matching operation unit 306 b receives the one-framedelayed signal d2 f and the two-frame delayed signal d4 f and carriesout pattern matching on vertically elongated sub-block areasconstituting parts of the pixel blocks in the respective frames.

From the vertical block matching quantities vp11, vp12, vp13 output byvertical block matching operation unit 306 a, matching horizontal highfrequency component extractor 307 a obtains a matching horizontal highfrequency component hebpm1. From the vertical block matching quantitiesvp21, vp22, vp23 output by vertical block matching operation unit 306 b,matching horizontal high frequency component extractor 307 b obtains amatching horizontal high frequency component hebpm2.

The two matching horizontal high frequency components hebpm1 and hebpm2calculated by the matching horizontal high frequency componentextractors 307 a, 307 b are sent to the horizontal high frequencymaximum value selector 323. The horizontal high frequency maximum valueselector 323 selects the largest of the two matching horizontal highfrequency components hebpm1 and hebpm2 and outputs it as the matchinghorizontal high frequency component hebpm.

This matching horizontal high frequency component hebpm is output to thethird moving block coefficient converter 313 as a value indicating thepresence of a horizontal motion component in the pixel blocks over atwo-frame interval.

Instead of performing pattern matching on the one-frame delayed signald2 f and two-frame delayed signal d4 f, the horizontal block matchingoperation unit 303 b and vertical block matching operation unit 306 bmay, like the block matching operation unit 301 b, be configured toperform pattern matching on the current frame signal Di0 and thetwo-frame delayed signal d4 f, which still enables the vertical highfrequency maximum value selector 322 and horizontal high frequencymaximum value selector 323 to obtain absolute difference results(matching quantities) over a two-frame interval.

The structure and operation of the first, second, and third moving blockcoefficient conversion units 311, 312, 313 are the same as those shownin FIGS. 2A, 2D, 2E, and 2F, so that a description will be omitted. Thepattern matching detector 31 in FIG. 20A carries out pattern matching onpixel blocks in the one-frame delayed signal d2 f and current framesignal Di0 and calculates a similarity, carries out pattern matching onpixel blocks in the one-frame delayed signal d2 f and two-frame delayedsignal d4 f and calculates a similarity, calculates a block matchingquantity blkpm, also obtains high frequency vertical and horizontalcomponents of the absolute difference values, and generates moving blockcoefficients dfmat, cbmat from these components and the block matchingquantity blkpm, taking patterns of vertical and horizontal pixel blockmotion into account.

Referring again to FIG. 19, the current frame signal Di0, one-framedelayed signal d2 f, and two-frame delayed signal d4 f are input to theframe difference detector 20 b together with the moving blockcoefficient dfmat from the pattern matching detector 31 and edgedecision coefficients efkm from the edge detector 40.

The frame difference detector 20 b obtains one-frame and two-framedifferences, but in obtaining these differences it selects an extent ofpixels according to the moving block coefficient dfmat and edge decisioncoefficients efkm and carries out a smoothing process with the selectedextent. It then detects the smoothed frame differences, combines the twoframe differences obtained as results, and outputs the combined framedifference signal sfdiff to the motion information corrector 50.

The current frame signal Di0 and one-frame delayed signal d2 f are inputto the frame difference calculator 21 c in the frame difference detector20 b, together with the moving block coefficient dfmat from the patternmatching detector 31 and the edge decision coefficients efkm from theedge detector 40. The current frame signal Di0 and the two-frame delayedsignal d4 f are input to the frame difference calculator 21 d, togetherwith the moving block coefficient dfmat from the pattern matchingdetector 31 and the edge decision coefficients efkm from the edgedetector 40.

The frame difference calculator 21 c obtains a frame difference frd1indicating one-frame differences between the current frame signal Di0and the one-frame delayed signal d2 f. The frame difference calculator21 d obtains a frame difference frd2 indicating two-frame differencesbetween the current frame signal Di0 and the two-frame delayed signal d4f. Frame difference frd1 is identical to the frame difference frddescribed in regard to the device in FIG. 1.

Nonlinear conversion unit 22 c performs a nonlinear conversion on framedifference frd1 to obtain a frame difference signal fdiff1. Nonlinearconversion unit 22 d performs a nonlinear conversion on frame differencefrd2 to obtain a frame difference signal fdiff2. Both frame differencesignals fdiff1, fdiff2 are output to the difference combiner 23.

The difference combiner 23 receives the one-frame difference signalfdiff1 representing one-frame differences from the nonlinear conversionunit 22 c and the two-frame difference signal fdiff2 representingtwo-frame differences from the nonlinear conversion unit 22 d, combinesthem by selecting the largest of the two difference signal valuesfdiff1, fdiff2, and outputs the resulting combined frame differencesignal sfdiff, the frame differences in which represent motion in thevideo signal.

The combining method used by the difference combiner 23 is not limitedto taking the largest value; the difference combiner 23 may calculatethe mean value, or may perform a combining computation with weightingcoefficients based on the magnitudes of the input one-frame differencesignal fdiff1 and two-frame difference signal fdiff2.

When calculating the frame difference signal sfdiff as described above,the frame difference detector 20 b selects an extent of pixels on whichto carry out a smoothing process according to the moving blockcoefficient dfmat and edge decision coefficients efkm, thereby obtaininga frame difference signal that has been smoothed within an appropriateextent responsive to pattern motion and edge direction. Since it alsoobtains two-frame differences, it can output a more accurate framedifference signal sfdiff.

The frame difference signal sfdiff is sent to the motion informationcorrector 50 and a motion detection signal mds is obtained by the sameconfiguration of elements as shown in FIG. 1 in the first embodiment. Adescription will be omitted.

In the motion detection device 2 in FIG. 19 as described above, thepattern matching detector 31 generates the moving block coefficientsdfmat, cbmat indicating pattern motion in pixel blocks by patternmatching on pixel blocks in the signal one frame before and the signaltwo frames before as well as on the current frame signal and the signalone frame before. By selecting the extent of pixels on which to performsmoothing processing according to the generated moving block coefficientdfmat and edge decision coefficients efkm, the frame difference detector20 b obtains one-frame differences and two-frame differences that havebeen smoothed within appropriate extents responsive to edge directionand pattern motion in the image.

Even in an image with noise, accordingly, the motion detection signalmds obtained from the motion detection device 2 indicates video motionwith high accuracy, preventing motion from being mistakenly detected ormissed. By calculating and combining one-frame differences and two-framedifferences, the motion detection device 2 can respond to high-speedimage motion, preventing such motion from being missed.

The steps by which one-frame differences and two-frame differences inthe video signal are detected by use of moving block coefficientsobtained by pattern matching and edge decision coefficients obtained asa result of edge detection and by which a motion detection signal mdsindicating a degree of motion is output in the video signal processingdevice in the second embodiment are substantially as described withreference to FIGS. 17 and 18 in the first embodiment, so that a fulldescription will be omitted, but in the second embodiment, the framedelayed signal input in step S101 in FIG. 17 becomes a pair of signalsincluding the one-frame delayed signal d2 f and the two-frame delayedsignal d4 f. In the block matching in step S102, in generating movingblock coefficients dfmat, cbmat indicating pixel block motion, thepattern matching detector 31 calculates two degrees of patternsimilarity, one for pixel blocks in the one-frame delayed signal d2 fand the current frame signal Di0 and another for pixel blocks in theone-frame delayed signal d2 f and the two-frame delayed signal d4 f. Theframe difference signal in step S108 in FIG. 18 is obtained by combiningone-frame and two-frame differences.

The steps other than the above steps S101, S102, and S108 are carriedout as described in the flowchart in FIGS. 17 and 18.

In the video signal processing device in the second embodiment, asdescribed above, the pattern matching detector 31 generates the movingblock coefficients dfmat, cbmat indicating pixel block pattern motion bypattern matching on pixel blocks in the signal one frame before and thesignal two frames before, as well as on the current frame signal and thesignal one frame before, and by selecting the extent of pixels on whichto perform smoothing processing according to the generated moving blockcoefficient dfmat and edge decision coefficients efkm. The framedifference detector 20 b obtains one-frame differences and two-framedifferences that have been smoothed within appropriate extentsresponsive to edge direction and pattern motion in the image. The motioninformation corrector 50 then obtains a motion information signal md0that has been modified according to edge direction and pattern motion asindicated by the moving block coefficients cbmat and the edge decisioncoefficients cedkm.

Accordingly, since moving block coefficients are obtained over aninterval of two frames and both one-frame and two-frame differences areobtained, high-speed image motion can be processed, preventing themotion from being missed and a still image portion from being detectedby mistake. In addition, since the smoothing extent is switched on thebasis of edge direction and pattern motion in the image when the framedifferences are obtained, frame difference signals that have beensmoothed within appropriate extents can be obtained, and even when noiseis present in the image, motion detection can be carried out with highaccuracy, preventing motion from being mistakenly detected or missed.

Although a pixel P0 in the current frame signal Di0 is the pixel ofinterest for which motion is detected when the frame difference detector20 b detects one-frame differences and two-frame differences in thevideo signal processing device in the second embodiment, in general thepixel of interest P0 need not be disposed in the current frame; motiondetection may be centered on a frame of a signal that has been delayedby one or more fields or frames, and the pixel of interest P0 may bedisposed in that delayed signal. In that case, the one-frame andtwo-frame differences detected by the frame difference detector 20 b aredifferences involving the two frames preceding the central frame on thetime axis or the two frames following the central frame on the timeaxis. It is also possible to combine differences with frames disposed inboth the preceding and following directions. Provided the motiondetection signal is obtained by detecting differences between theseframes and the central frame, the same effects as in the secondembodiment are obtained.

The constituent units of the frame delay and motion detection sectionsmay be configured either as hardware or as software, in the form of aprogrammed computing device.

Third Embodiment

The video signal processing devices in the first and second embodimentsdetected motion from a video signal locally, and can be applied inmotion adaptive processing by processing the video signal according tothe detected motion.

As an example of this type of application, a motion adaptive scan lineinterpolation process that performs scan conversion to convert aninterlaced signal to a progressive signal (IP conversion) will bedescribed.

In the third embodiment described below, during IP conversion, a videosignal processing device generates an interpolated signal by switchingbetween inter-field interpolation and intra-field interpolationaccording to the motion detection signal that indicates the amount ordegree of motion detected. The motion detection device 1 in the videosignal processing device in the first embodiment is used to detectmotion.

FIG. 21 is a block diagram showing the configuration of a video signalprocessing device of the third embodiment of the invention (device forimplementing the video signal processing method of the third embodimentof the invention). The device is a motion adaptive processor 3 that usesthe motion detection device 1 to detect motion on the basis ofdifferences between frames in a video signal and interpolates scanninglines by processing responsive to the output of the motion detectiondevice 1 to generate a progressive signal. Constituent elements in FIG.21 that are identical to or correspond to elements in the firstembodiment shown in FIG. 1 have the same reference numerals.

In addition to using the motion detection device 1 (see FIG. 1) in thevideo signal processing device in the first embodiment, the motionadaptive processor 3 which is the video signal processing device in thethird embodiment has a pair of field memories 11, 12 constituting aframe delay unit 13 as in the first embodiment, a motion adaptiveinterpolator 600, and a rate doubler 601. Operating according to theoutput of the motion detection device 1, the motion adaptiveinterpolator 600 generates a motion adaptive scanning line interpolationsignal I and outputs it together with the received ‘real’ scanning linesignal R. The rate doubler 601 embeds the scanning line interpolationsignal I between corresponding lines in the received scanning linesignal R, doubles the scan rate, and outputs the resulting progressivevideo signal Do.

The input video signal Di0 is input sequentially to the motion adaptiveprocessor 3, which performs scanning line interpolation by motionadaptive processing and outputs the progressive signal Do.

A one-field delayed signal d1 f and a one-frame delayed signal d2 f areoutput from the frame delay unit 13.

The motion detection device 1 in the motion adaptive processor 3receives the current frame signal Di0, the one-frame delayed signal d2f, and the one-field delayed signal d1 f, detects motion in the videosignal between the current frame signal and the signal of the precedingframe, and outputs a motion detection signal mds indicating the degreeof motion detected.

The motion detection device 1 may be structured as shown in FIG. 1, andmay thus switch the extent of the smoothing processing performed in thedetermination of the difference between frames responsive to patternmotion and edge direction in the image, using moving block coefficientsobtained from pattern matching of pixel blocks in the current framesignal Di0 and the one-frame delayed signal d2 f and edge decisioncoefficients resulting from edge detection, thereby obtaining a framedifference signal that has been smoothed within an appropriate extent,and may further modify the frame difference signal responsive to patternmotion and edge direction to generate motion information modified so asto make motion easier to detect at edges and in patterns where movingareas tend to be misrecognized as still, thereby carrying out motiondetection with high accuracy and avoiding motion detection mistakes.

The motion adaptive interpolator 600 receives the current frame signalDi0, the one-frame delayed signal d2 f, the one-field delayed signal d1f, and the motion detection signal mds output from the motion detectiondevice 1.

The field into which scanning lines are interpolated in FIG. 21 (thefield undergoing interpolation) is the field associated with theone-field delayed signal d1 f. The motion detection device 1 detectsmotion between the current frame signal Di0 and the one-frame delayedsignal d2 f, and the motion detection signal mds indicates the degree ofmotion detected, so that interpolating scanning lines into the one-fielddelayed signal d1 f is equivalent to generating an interpolation signalat the center of the detected motion. In FIG. 5, for example, theinterpolation signal interpolates a pixel on the scanning line n thatpasses midway between pixels Pf1 a and Pf1 b in the one-field delayedsignal d1 f.

The motion adaptive interpolator 600 generates the scanning lineinterpolation signal I by performing motion adaptive processing based onthe motion detection signal mds output from the motion detection device1 for the one-field delayed signal d1 f, the current frame signal Di0,and the one-frame delayed signal d2 f, and outputs the scanning lineinterpolation signal I together with the received scanning line signal Rin the one-field delayed signal d1 f. The processing that generates thescanning line interpolation signal I is carried out with respect to theone-field delayed signal d1 f. Depending on the value of the motiondetection signal mds, the motion adaptive interpolator 600 performsinter-field interpolation by embedding the pixels of the correspondingscanning lines of the one-frame delayed signal d2 f, which temporallyprecedes the one-field delayed signal d1 f, or the current frame signalDi0, which temporally follows the one-field delayed signal d1 f, orperforms intra-field interpolation by using the signals of the pixels ofthe one-field delayed signal d1 f vertically above and below.

The motion adaptive interpolator 600 generates the scanning lineinterpolation signal I by, for example, mixing the signal obtained byinter-field interpolation with the signal obtained by intra-fieldinterpolation according to the value of the motion detection signal mds.That is, when the motion detection signal mds indicates that the localimage is definitely still (mds=0), inter-field interpolation is carriedout; when the value of the motion detection signal mds is large enoughto indicate definite motion, intra-field interpolation is carried out;when the value of the motion detection signal mds is intermediatebetween zero and the value indicating definite motion, the inter-frameinterpolated signal and the intra-frame interpolated signal are mixed ina ratio responsive to the value of the motion detection signal mds togenerate the scanning line interpolation signal I.

Inter-field interpolation may be carried out by use of just one of theone-frame delayed signal d2 f and the current frame signal Di0: forexample, by use only of the temporally preceding one-frame delayedsignal d2 f.

The field undergoing interpolation need not be the field of theone-field delayed signal dif as described above; it may be the field ofthe current frame signal Di0 or the field of the one-frame delayedsignal d2 f.

Next the rate doubler 601 receives the scanning line interpolationsignal I and the received scanning line signal R output from the motionadaptive interpolator 600, embeds the interpolation signal betweencorresponding scanning lines in the received scanning line signal,doubles the scan rate to convert the resulting signal to the progressivevideo signal Do, and outputs the progressive video signal Do. Theprogressive video signal Do output from the rate doubler 601 is therebyoutput from the motion adaptive processor 3.

As described above, in the video signal processing device in the thirdembodiment, the motion detection device 1 in the first embodiment isused to obtain a motion detection signal mds that has been obtained froma frame difference signal that has been smoothed over a suitable extentand modified according to edge direction and pattern motion, and themotion adaptive interpolator 600 carries out a scanning lineinterpolation process by means of motion adaptive processing responsiveto the motion detection signal mds, thereby obtaining a progressivesignal Do.

Therefore, even when noise is present in the image, it is possible toperform motion detection with high accuracy, avoiding the mistakes offalse detection of nonexistent motion and failure to detect real motion.Since moving parts are not detected as still; this motion detectionresult can be used to obtain a progressive signal with less picturequality degradation due to problems such as flicker, blur, combing, andjitter than before in the result of line interpolation by motionadaptive processing.

Although the motion adaptive processor 3 in the third embodiment shownin FIG. 21 is configured to detect motion in the video signal by use ofthe motion detection device 1 in FIG. 1, it may be configured to use themotion detection device 2 in the video signal processing device in thesecond embodiment (see FIG. 19) by adding another frame delay unit 15 toobtain a two-frame delayed signal d4 f as shown in FIG. 22. In thiscase, since moving block coefficients are obtained over a two-frameinterval and since both one-frame and two-frame differences areobtained, high-speed image motion can be detected and thus notmisidentified as non-motion, and the results of this more accuratemotion detection can be used to perform scanning line interpolation bymotion adaptive processing, thereby obtaining a progressive signal Do.

The motion adaptive processor 3 shown in FIG. 21 and the motion adaptiveprocessor 3 b shown in FIG. 22 in the third embodiment carry out motionadaptive scanning line interpolation of a video signal, but in motioncompensating scanning line interpolation processing, which detectsmotion vectors and performs scan conversion according to the motionvectors, the result of motion detection by the motion detection device 1or 2 can be used as a parameter in the motion compensation processing,so that the motion detection result is used together with the motionvectors. Scanning line interpolation processing can then be carried outby highly accurate motion compensation to obtain a progressive signalDo.

The constituent units of the motion adaptive processor 3 described abovemay be configured either as hardware or as software, in the form of aprogrammed computing device.

Fourth Embodiment

As a next exemplary application to motion adaptive processing, a case inwhich motion adaptive processing is carried out in three-dimensionalnoise removal will be shown.

In the fourth embodiment, the motion detection device 1 in the videosignal processing device in the first embodiment is used to detectmotion in a motion adaptive video signal processing device that, incarrying out three-dimensional noise removal, controls the noise removaleffect according to a motion detection signal indicating the amount ordegree of motion detected.

FIG. 23 is a block diagram showing the configuration of a video signalprocessing device of the fourth embodiment of the invention (device forimplementing the video signal processing method of the fourth embodimentof the invention). The device is a motion adaptive processor 4 thatdetects motion on the basis of differences between frames in a videosignal and multiplies the noise component of the signal by an IIRcoefficient obtained according to the output of the motion detectiondevice, thereby performing frame recursive noise reduction. Constituentelements in FIG. 23 that are identical to or correspond to elements inthe first embodiment shown in FIG. 1 have the same reference numerals.

In addition to using the motion detection device 1 (see FIG. 1) in thevideo signal processing device in the first embodiment to detect motion,the motion adaptive processor 4 which is the video signal processingdevice in the fourth embodiment has a frame delay unit 13, a frame delayunit 603, and a motion adaptive noise reducer 610. The motion adaptivenoise reducer 610 obtains an IIR noise component coefficient accordingto the output of the motion detection device 1, and performs framerecursive noise reduction processing.

The motion adaptive noise reducer 610 includes a noise extractor 611, acoefficient generator 612 that generates the IIR coefficient responsiveto the motion detection result, a multiplier 613 that multiples thenoise component output from the first field memory 11 by the IIRcoefficient to output a noise recursion quantity, and an operation unit614 that adds (or subtracts) the noise recursion quantity Nd from themultiplier 613 to (or from) the input video signal Di0.

An interlaced signal is input sequentially to the motion adaptiveprocessor 4 as the input video signal Di0. After carrying out noisereduction processing, the motion adaptive processor 4 outputs anoise-reduced signal Dnro.

The frame delay units 13, 603 are memory units that delay the videosignal by one frame each. The first frame delay unit 13 delays the inputvideo signal Di0 by one frame and outputs a one-frame delayed signal d2f; the second frame delay unit 603 delays the noise-reduced signal Dnroby one frame and outputs a one-frame delayed low noise signal ln2. Whenan interlaced signal is input, the frame delay units 13, 603 may beconfigured from pairs of field memories that impart a one-field delay,and a one-field delayed signal d1 f may be generated in the frame delayunit 13 and supplied to the motion detection device 1, as shown. Eachframe delay unit 13, 603 is identical in configuration and operation toframe delay unit 13 in FIG. 1.

The motion detection device 1 receives the current frame signal Di0,also receives the one-frame delayed signal d2 f and the one-fielddelayed signal d1 f from frame delay unit 13, detects motion in thevideo signal between the current frame signal Di0 and the signal d2 f ofthe preceding frame, and outputs a motion detection signal mdsindicating the amount or degree of motion detected. The motion detectiondevice 1 has the structure shown in FIG. 1. It switches the extent ofthe smoothing processing performed in the determination of thedifference between frames responsive to pattern motion and edgedirection in the image, using moving block coefficients obtained frompattern matching and edge decision coefficients resulting from edgedetection, thereby obtaining a frame difference signal that has beensmoothed within an appropriate extent, then modifies the framedifference signal responsive to pattern motion and edge directions andobtains modified motion information that facilitates motion detection atedges and in patterns where moving areas tend to be misrecognized asstill, thereby carrying out motion detection with high accuracy andavoiding motion detection mistakes.

The current frame signal Di0, the one-frame delayed low noise signal ln2from frame delay unit 603, and the motion detection signal mds from themotion detection device 1 are input to the motion adaptive noise reducer610. From the frame difference values between the current frame signalDi0 and the one-frame delayed low noise signal ln2, the motion adaptivenoise reducer 610 extracts the noise component, determines a noisecomponent IIR coefficient responsive to the motion detection signal mdsfrom the motion detection device 1, uses the extracted noise componentand the IIR coefficient to carry out a noise reduction process, andoutputs the noise-reduced signal Dnro. The noise-reduced signal Dnro,which is a signal of the same frame as the current frame signal Di0, issent to the second frame delay unit 603.

In the noise extractor 611 in the motion adaptive noise reducer 610, forexample, the current frame signal Di0 is subtracted from the one-framedelayed signal ln2 to obtain frame differences Diff between the currentframe signal Di0 and the one-frame delayed signal ln2. The amplitude ofthe differences is then limited to within a predetermined value (e.g.,within±dTh) and the amplitude limited differences are output as thenoise component Dfn. The frame differences Diff include both motion andnoise components in the video signal. Accordingly, when the noisecomponent Dfn obtained from the frame differences Diff is zero, therelevant part of the video signal has no motion or noise, but as themotion or noise increases, the Dfn value increases. The noise componentDfn obtained by the noise extractor 611 is output to the multiplier 613as the noise component of the current frame signal Di0.

The coefficient generator 612 receives the motion detection signal mdsfrom the motion detection device 1 and generates an IIR coefficient Kmresponsive to the motion detection signal mds. The IIR coefficient Kmgenerated by the coefficient generator 612 has a value in the range 0Km<1. This value sets the recursion level in the recursive noisereduction process. The closer the value of the IIR coefficient is toone, the greater the recursion level and the noise reduction effectbecome. If the IIR coefficient Km is zero (Km=0), the recursion level iszero and noise reduction is not performed.

The coefficient generator 612 accordingly generates the IIR coefficientKm by a multiplication process, for example, or by use of an LUT storedin a read-only memory (ROM) so that Km varies in response to the degreeof motion indicated by the motion detection signal mds from the motiondetection device 1. When the motion detection signal mds has a largevalue, indicating definite motion, the IIR coefficient is set to zero(Km=0). When the motion detection signal mds is zero, indicating a stillimage or noise component, Km is set to its maximum value Kmax (Kmax<1).When the motion detection signal mds has a value intermediate betweenzero and ‘definite motion’, the value of the IIR coefficient Km variesaccordingly, becoming smaller as the mds value becomes larger. An IIRcoefficient that takes account of motion is thereby obtained.

The IIR coefficient which thus varies between zero and Kmax is outputfrom the coefficient generator 612 to the multiplier 613.

The multiplier 613 calculates the noise recursion quantity Nd bymultiplying the noise component Dfn by the IIR coefficient Km(Nd=Km×Dfn). The noise recursion quantity Nd thus obtained is output tothe operation unit 614.

When the motion detection signal mds indicates motion, the IIRcoefficient Km, which is calculated from the mds value, is output aszero, so that the noise recursion quantity Nd is zero and no noisereduction is performed. When a pixel in a still part of the image isdetected, Km has its maximum value Kmax, so that the value of the noiserecursion quantity Nd is set to be as large as possible, enablingmaximum noise reduction to be carried out. Between these extremes, thenoise reduction effect varies gradually responsive to the amount ofmotion indicated by the motion detection signal.

The operation unit 614 adds (or subtracts) the noise recursion quantityNd to (or from) the current frame signal Di0 to reduce noise in thevideo signal, thereby obtaining the noise-reduced signal Dnro. Theoperation unit 614 either adds or subtracts according to the sign(positive or negative) of the noise recursion quantity Nd so as toreduce noise.

Next, the noise-reduced signal Dnro output from the motion adaptivenoise reducer 610, which is a signal for the same frame as the inputvideo signal, is sent to frame delay unit 603.

As described above, the video signal processing device in the fourthembodiment uses the motion detection device 1 in the first embodiment toobtain a frame difference signal that has been appropriately smoothedand a motion detection signal mds that has been modified responsive toedge direction and pattern motion; the motion adaptive noise reducer 610then obtains an IIR coefficient responsive to the motion detectionsignal mds and applies it to the noise component to carry out a framerecursive noise reduction process, thereby performing a motion adaptiveprocess in which the noise reduction effect is controlled by the resultof motion detection.

Accordingly, even when noise is present in the image, motion detectioncan be carried out with high accuracy, preventing motion from beingmistakenly detected or missed, so that in the noise-reduced video signalobtained by motion adaptive processing, noise is reduced in still partsof the image without producing trails or ghosts in moving parts, and thenoise-reduced signal is comparatively free of defects such as flickerand blur.

Although the motion adaptive processor 4 in the fourth embodiment shownin FIG. 23 is configured to detect motion in the video signal by use ofthe motion detection device 1 in FIG. 1, it may be configured to use themotion detection device 2 in the video signal processing device in thesecond embodiment (see FIG. 19) by adding another frame delay unit 15 toobtain a two-frame delayed signal d4 f. In this case, since moving blockcoefficients are obtained over a two-frame interval and since bothone-frame and two-frame differences are obtained, high-speed imagemotion can be detected and thus not misidentified as non-motion, and theresults of this more accurate motion detection can be used to performnoise reduction by a motion adaptive process in which the noisereduction effect is controlled by the motion detection result.

Although the video signal processing device in the fourth embodiment wasdescribed as receiving an interlaced input video signal, if the inputvideo signal is a progressive signal, the motion detection and noisereduction processes can be performed with substantially the sameconfiguration. When a progressive signal is input, the out-of-phaseone-field delayed signal dif is not obtained and the frame delay unit 13outputs only the one-frame delayed signal d2 f obtained by delaying thecurrent frame signal Di0 by one frame, so that the motion detectiondevice 1 is configured to perform motion detection using only in-phaseframe signals.

Whereas the motion adaptive processor 4 in the fourth embodiment wasprovided with frame delay units 13 and 603 configured to obtain aone-frame delayed signal d2 f for the motion detection device 1 and aone-frame delayed low noise signal ln2 for the motion adaptive noisereducer 610, in a variation of the fourth embodiment, the motionadaptive processor 4 b has only one frame delay unit 603, which delaysthe noise-reduced signal Dnro by one frame to obtain a one-frame delayedlow noise signal ln2 for input to the motion adaptive noise reducer 610and by one field to obtain a one-field delayed low noise signal ln1 forinput to the motion detection device 1.

The configuration of the motion adaptive noise reducer 610 in the motionadaptive processor 4 in the fourth embodiment is not limited to theconfiguration shown in FIG. 23, in which the coefficient generator 612generates an IIR coefficient responsive to the motion detection signalmds generated by the motion detection device 1, so that the IIRcoefficient applied to the noise component varies according to themotion detection result. In another variation of the fourth embodiment,the IIR coefficient has a fixed value, and the input video signal ismixed with the signal resulting from fixed-strength recursive noisereduction in a mixing ratio that varies responsive to the motiondetection signal mds generated by the motion detection device 1. Theresulting mixed signal is again a noise-reduced signal in whichrecursive noise reduction is controlled according to motion, so thateffects similar to those of the fourth embodiment shown in FIG. 23 areobtained.

In another variation of the fourth embodiment, instead of performingrecursive noise reduction as in FIG. 23, the motion detection result isapplied to non-recursive or finite impulse response (FIR) noisereduction as in FIG. 25. Constituent elements of FIG. 25 that areidentical to or correspond to constituent elements of FIG. 23 have thesame reference characters. The motion adaptive processor 4 c in FIG. 25replaces the motion adaptive noise reducer 610 of FIG. 23 with adifferent motion adaptive noise reducer 620, adds a frame delay unit 15to obtain a two-frame delayed signal d4 f, and uses the motion detectiondevice 2 of the second embodiment. The motion adaptive noise reducer 620includes a filtering section 621 and a motion mixer 622.

The motion detection device 2 and the frame delay units 13, 15 areconfigured and operate as in the second embodiment, so that adescription will be omitted.

The coefficient generator 612 in the motion adaptive noise reducer 620performs noise reduction filtering on the current frame signal Di0, theone-frame delayed signal d2 f, and the two-frame delayed signal d4 f (byprocessing the three signals concurrently, for example) to obtain afiltered signal dfir. The motion mixer 622 mixes the filtered signaldfir with one of the unfiltered signals, e.g., the current frame signalDi0, in a mixing ratio responsive to the motion detection signal mdsoutput from the motion detection device 2 to obtain the noise-reducedsignal Dnro.

When the motion detection signal mds indicates motion, the motion mixer622 uses a mixing ratio in which the current frame signal Di0 has a highproportion. When the motion detection signal mds is zero, indicating astill image or noise component, the motion mixer 622 uses a mixing ratioin which the filtered signal dfir has a high proportion.

When the value of the motion detection signal mds is intermediatebetween zero and the ‘motion’ value, the mixing ratio is varied so thatthe proportion of the filtered signal dfir decreases as the mds valueincreases.

A noise-reduced signal Dnro is thereby obtained in which FIR noisereduction has been carried out with motion taken into consideration. Thesame effects as in FIG. 23 are accordingly obtained.

If the coefficients used in the filtering section 621 in the motionadaptive noise reducer 620 are varied according to the motion detectionsignal mds output from the motion detection device 2, the motion mixer622 may be omitted, still obtaining a noise-reduced signal Dnro in whichFIR noise reduction has been carried out with motion taken intoconsideration.

The constituent units of the motion adaptive processors 4, 4 b, 4 cdescribed above may be configured either as hardware or as software, inthe form of a programmed computing device.

Fifth Embodiment

Video signal processing devices in which the video signal processingmethods can be carried out have been described in the first to fourthembodiments, but the invention can also be applied in a video displaydevice that detects motion in an input video signal, performs motionadaptive processing, and generates a high-quality display. The videodisplay device described in the fifth embodiment below processes a TVbroadcast signal or a video signal input from a recording andreproduction device such as a DVD or VTR or a TV broadcast receiver etc.and displays the video signal using the motion adaptive processor 3 or 4of the video signal processing device in the third or fourth embodiment.

FIG. 26 is a block diagram showing an exemplary configuration of thevideo display device in the fifth embodiment of the invention. Theillustrated video display device 5 has a video signal processing device500 configured in the same way as the motion adaptive processor 3 in thethird embodiment (or the motion adaptive processor 4 in the fourthembodiment). The video display device 5 is also provided with an inputterminal 501, an input signal processor 502, a display processor 503,and a display unit 504, enabling it to display a video signal by motionadaptive processing. Aside from these additional parts 501, 502, 503,504, the video display device 5 is configured and operates as in thefirst to fourth embodiments; a detailed description will be omitted.

The TV broadcast signal or the video signal from a recording,reproducing, or receiving device such as a video tape recorder (VTR), adigital versatile disc (DVD) player, or a TV broadcast receiver is inputat the input terminal 501. The signal input at the input terminal 501 issent to the input signal processor 502.

The input signal processor 502 processes of the TV broadcast signal orvideo signal from the recording, reproducing, or receiving device (VTR,DVD player, TV broadcast receiver, etc.) by, for example, converting thesignal to a digital signal, if an analog signal is input, separatingsynchronizing signals, decoding MPEG data when MPEG data are received,and so on. After this input signal processing, the video signal is sentto the video signal processing device 500.

The video signal resulting from the input signal processing may beeither an interlaced signal or a progressive signal.

Besides frame-delaying the video signal from the input signal processor502 and performing motion detection by use of frame differences, thevideo signal processing device 500 carries out motion adaptive noisereduction processing process and/or motion adaptive scanning lineinterpolation processing. When the input signal is interlaced, the videosignal processing device 500 carries out three dimensional noisereduction and IP conversion, using the motion detection results in bothprocesses. Alternatively, only IP conversion may be carried out. Whenthe input signal is progressive, only noise reduction processing iscarried out. The operation of the video signal processing device 500 inmotion detection, motion adaptive noise reduction, and motion adaptiveIP conversion are as described in the first to fourth embodiments;details will be omitted.

After motion adaptive processing in the video signal processing device500, the video signal is input to the display processor 503. The displayprocessor 503 carries out the signal processing necessary for conversionto a picture signal, such as scaling, conversion of the gradation scaleof the video signal by a gray scale correction process, etc., andoutputs the result for display by the display unit 504.

The display unit 504 displays a video picture based on the picturesignal from the display processor 503.

Since, as described above, the video display device in the fifthembodiment uses the video signal processing devices and video signalprocessing methods of the first to fourth embodiments, even in pictureswith small amounts of motion or noise, it does not mistake still partsof the picture for moving parts or moving parts for still parts, so thatits motion detection is highly accurate, motion is not missed, and avideo picture of high quality can be displayed, based on a video signalwith less picture quality degradation than before due to problems suchas flicker, blur, combing, and jitter arising from motion adaptiveprocessing.

Several variations of the preceding embodiments have been describedabove, but those skilled in the art will recognize that furthervariations are possible within the scope of the invention, which isdefined in the appended claims.

1. A motion detection device for detecting motion in a video signalincluding temporally differing first and second frames, the motiondetection device comprising: a pattern matching detector that calculatespattern similarity between a pixel block in the first frame and a pixelblock in the second frame and generates a moving block coefficientindicating block movement based on the similarity, the pixel block inthe first frame being centered on a pixel of interest at which motion isto be detected, the pixel block in the second frame being positioned ata pixel position corresponding to the pixel of interest; an edgedetector that detects edges in a vicinity of the pixel of interest fromthe video signal in the first and second frames and generates at leastone edge decision coefficient indicating a degree of edge presence; aframe difference detector that selects, based on the moving blockcoefficient and the edge decision coefficient, an extent of horizontallyaligned pixels including the pixel of interest or an extent ofvertically aligned pixels including the pixel of interest, performs asmoothing process within the selected extent, and performs aframe-to-frame difference calculation before or after the smoothingprocess, thereby generating a frame difference signal for the pixel ofinterest; and a motion information corrector that generates motioninformation for the pixel of interest from the frame difference signalgenerated by the frame difference detector.
 2. The motion detectiondevice of claim 1, wherein the pattern matching detector comprises: ablock matching operation unit that takes absolute differences betweenpixel values of pixels in corresponding positions in the pixel block inthe first frame and the pixel block in the second frame, calculates asum of the absolute differences at all pixel positions in the first andsecond blocks, and generates a value based on the sum of the absolutedifferences as a value of the similarity; and a moving block coefficientconversion unit that obtains the moving block coefficient by convertingthe value of the similarity generated by the block matching operationunit.
 3. The motion detection device of claim 2, wherein the patternmatching detector further comprises: a matching vertical high frequencycomponent generator that generates a matching vertical high frequencycomponent by dividing the pixel block vertically into a plurality ofsub-blocks, taking a sub-block sum of the absolute differences in eachsub-block, and taking differences between the sub-block sums ofvertically adjacent sub-blocks; and a matching horizontal high frequencycomponent generator that generates a matching horizontal high frequencycomponent by dividing the pixel block vertically into a plurality ofsub-blocks, taking a sub-block sum of the absolute differences in eachsub-block, and taking differences between the sub-block sums ofhorizontally adjacent sub-blocks; wherein the moving block coefficientconversion unit converts the value of the similarity generated by theblock matching operation unit to generate the motion block coefficientaccording to values of the matching vertical high frequency componentgenerated by the matching vertical high frequency component generator orthe matching horizontal high frequency component generated by thematching horizontal high frequency component generator.
 4. The motiondetection device of claim 1, wherein the edge detector comprises: avertical edge detection section that extracts a vertical high frequencycomponent in the vicinity of the pixel of interest in the first frame,extracts a vertical high frequency component in a vicinity of the pixelposition in the second frame corresponding to the pixel of interest, andgenerates a vertical edge decision coefficient indicating a degree ofvertical edge presence from the extracted vertical high frequencycomponents; and a horizontal edge detection section that extracts ahorizontal high frequency component in the vicinity of the pixel ofinterest in the first frame, extracts a horizontal high frequencycomponent in the vicinity of the pixel position in the second framecorresponding to the pixel of interest, and generates a horizontal edgedecision coefficient indicating a degree of horizontal edge presencefrom the extracted horizontal high frequency components; the at leastone edge decision coefficient including both the vertical edge decisioncoefficient and the horizontal edge decision coefficient.
 5. The motiondetection device of claim 1, wherein the frame difference detectorcomprises: a subtractor that obtains a difference between the signal ofthe pixel of interest in the first frame and the signal of the pixel atthe same position as the pixel of interest in the second frame; a firstsmoothing processor that smoothes the difference obtained by thesubtractor in a first direction, the first direction being one of ahorizontal direction and a vertical direction; a first mixer that mixesthe signal smoothed by the first smoothing processor and the differenceobtained by the subtractor in a ratio responsive to at least one of thecoefficients generated by the pattern matching detector and the edgedetector; a second smoothing processor that smoothes the output of thefirst mixer in a second direction, the second direction being anotherone of the horizontal direction and the vertical direction; and a secondmixer that mixes the signal smoothed by the second smoothing processorand the output of the first mixer in a ratio responsive to at least oneof the coefficients generated by the pattern matching detector and theedge detector; the frame difference signal being generated from theoutput of the second mixer.
 6. The motion detection device of claim 1,wherein the frame difference detector comprises: a first smoothingprocessor that smoothes the signal of the pixel of interest in the firstframe in a first direction, the first direction being one of ahorizontal direction and a vertical direction; a first mixer that mixesthe signal smoothed by the first smoothing processor and the signal ofthe pixel of interest in the first frame in a ratio responsive to atleast one of the coefficients generated by the pattern matching detectorand the edge detector; a second smoothing processor that smoothes theoutput of the first mixer in a second direction, the second directionbeing another one of the horizontal direction and the verticaldirection; a second mixer that mixes the signal smoothed by the secondsmoothing processor and the output of the first mixer in a ratioresponsive to at least one of the coefficients generated by the patternmatching detector and the edge detector; a third smoothing processorthat smoothes the signal of the pixel position corresponding to thepixel of interest in the second frame in the first direction; a thirdmixer that mixes the signal smoothed by the third smoothing processorand the signal of the pixel corresponding to the pixel of interest inthe second frame in a ratio responsive to at least one of thecoefficients generated by the pattern matching detector and the edgedetector; a fourth smoothing processor that smoothes the output of thethird mixer in the second direction; a fourth mixer that mixes thesignal smoothed by the fourth smoothing processor and the output of thethird mixer in a ratio responsive to at least one of the coefficientsgenerated by the pattern matching detector and the edge detector; and asubtractor that takes a difference between the output of the secondmixer and the output of the fourth mixer; the frame difference signalbeing generated from the output of the subtractor.
 7. The motiondetection device of claim 1, wherein the motion information correctorgenerates the motion information of the pixel of interest by modifyingthe frame difference signal generated by the frame difference detectoraccording to the moving block coefficient generated by the patternmatching detector and the edge decision coefficient generated by theedge detector.
 8. The motion detection device of claim 7, wherein themotion information corrector generates the motion information for thepixel of interest by calculating a conversion value that variesaccording to the moving block coefficient and the edge decisioncoefficient, and modifying the frame difference signal generated by theframe difference detector by adding the conversion value to the framedifference signal or multiplying the frame difference signal by theconversion value.
 9. The motion detection device of claim 8, wherein theat least one edge decision coefficient includes a vertical edge decisioncoefficient and a horizontal edge decision coefficient and the motioninformation corrector comprises: a horizontal line motion adjuster thatcalculates a first conversion value having a value that variesresponsive to the moving block coefficient and the vertical edgedecision coefficient, and adds the calculated first conversion value tothe frame difference signal; and a vertical correlation adjuster thatcalculates a second conversion value having a value that variesresponsive to the moving block coefficient, the vertical edge decisioncoefficient, and the horizontal edge coefficient, and adds thecalculated second conversion value to the output of the horizontal linemotion adjuster or multiplying the output of the horizontal line motionadjuster by the second conversion value; the motion informationcorrector that outputs the output of the vertical correlation adjusteras the motion information.
 10. The motion detection device of claim 1,wherein a motion detection signal is generated from the motioninformation.
 11. A video signal processing device for performing motionadaptive scanning line interpolation based on the motion detectionsignal output by the motion detection device of claim 10 to convert aninterlaced scanning video signal to a progressive video signal, thevideo signal processing device comprising: a motion adaptiveinterpolator that receives the motion detection signal output from themotion detection device and generates a scanning line interpolationsignal responsive to a result of motion detection for each pixel; and arate doubler that generates the progressive video signal based on thescanning line interpolation signal generated by the motion adaptiveinterpolator.
 12. A video signal processing device for performing threedimensional noise reduction based on the motion detection signal outputby the motion detection device of claim 10 to eliminate noise componentslacking frame-to-frame correlation from the video signal, comprising amotion adaptive noise reducer that receives the motion detection signaloutput from the motion detection device and controls the noise reductioneffect.
 13. A video display device comprising: the video signalprocessing device of claim 11; a display unit that displays a videopicture; and a display processor that causes the display unit to displaythe video picture responsive to the video signal output by the videosignal processing device.
 14. A video display device comprising: thevideo signal processing device of claim 12; a display unit that displaysa video picture; and a display processor that causes the display unit todisplay the video picture responsive to the video signal output by thevideo signal processing device.
 15. A motion detection method fordetecting motion in a video signal including temporally differing firstand second frames, the motion detection method comprising: a patternmatching detection step for calculating pattern similarity between apixel block in the first frame and a pixel block in the second frame andgenerating a moving block coefficient indicating block movement based onthe similarity, the pixel block in the first frame being centered on apixel of interest at which motion is to be detected, the pixel block inthe second frame being positioned at a pixel position corresponding tothe pixel of interest; an edge detection step for detecting edges in avicinity of the pixel of interest from the video signal in the first andsecond frames and generating an edge decision coefficient indicating adegree of edge presence; a frame difference detection step forselecting, based on the moving block coefficient and the edge decisioncoefficient, an extent of horizontally aligned pixels including thepixel of interest or an extent of vertically aligned pixels includingthe pixel of interest, performing a smoothing process within theselected extent, and performing a frame-to-frame difference calculationbefore or after the smoothing process, thereby generating a framedifference signal for the pixel of interest; and a motion informationcorrection step for generating motion information for the pixel ofinterest from the frame difference signal generated in the framedifference detection step.